Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use OPTE underneath Oxide services requiring external addresses #1330

Closed
bnaecker opened this issue Jun 30, 2022 · 2 comments
Closed

Use OPTE underneath Oxide services requiring external addresses #1330

bnaecker opened this issue Jun 30, 2022 · 2 comments
Labels
networking Related to the networking.
Milestone
MVP

Comments

@bnaecker
Copy link
Collaborator

We need a way to expose some Oxide services to an external network. The most salient example is the customer providing us with some IP address where they'd like to reach the Nexus public API and web console. This address is not in our control, it's part of the customer network. To integrate this addressing with the Oxide rack network, we'll need to encapsulate this traffic onto the rack network, deliver it to the sled hosting that Nexus instance, where it'll be decapsulated for delivery to the actual program. In this comment, Robert proposed using OPTE for the on-sled part of this. The full path would start at the switch, where the customer network traffic is encapsulated into the rack physical network. OPTE is the complement to that, undoing the encapsulation on the way in and encapsulating the replies.

This issue tracks adding addresses for services like Nexus using this pattern. There are several pieces of work here, to be fleshed out.

  • Adding an OPTE / xde device. We currently only use this for guest traffic, so we'll have to extend things a bit to support other kinds of endpoints.
  • Create and manage system VNIs for this traffic. We currently reserve a range of VNIs for system and guest traffic, and we'll need to manage the system VNIs for this traffic.
  • Potentially putting a VNIC over that. It's not clear if this will actually be needed. We do need it for guests currently, since the Viona driver that exposes a virtio-net device to guests only works over VNICs.
  • An API for providing Nexus with this new, external address and asking it to listen there.
  • There will be lots of work communicating this to the switch as well, such as the external address, VNI, and address of the physical sled to deliver the traffic to.
@bnaecker bnaecker added the networking Related to the networking. label Jun 30, 2022
@bnaecker bnaecker added this to the MVP milestone Feb 22, 2023
@bnaecker
Copy link
Collaborator Author

bnaecker commented May 9, 2023

@luqmana I think this was addressed by #2857 right?

@luqmana
Copy link
Contributor

luqmana commented May 10, 2023

@luqmana I think this was addressed by #2857 right?

Yup!

@luqmana luqmana closed this as completed May 10, 2023
leftwo pushed a commit that referenced this issue Jun 26, 2024
Update crucible, add DTrace crucible to host image
3a09cb9 
Added a new package, crucible-dtrace that pulls from buildomat a package
that contains a set of DTrace scripts.  These scripts are extracted into
the global zone at /opt/oxide/crucible_dtrace/

Update Crucible to latest includes these updates:
Clean up dependency checking, fixing space leak (#1372)
Make a DTrace package (#1367)
Use a single context in all messages (#1363)
Remove `DownstairsWork`, because it's redundant (#1371)
Remove `WorkState`, because it's implicit (#1370)
Do work immediately upon receipt of a job, if possible (#1366)
Move 'do work for one job' into a helper function (#1365)
Remove `DownstairsWork` from map when handling it (#1361)
Using `block_in_place` for IO operations (#1357)
update omicron deps; use re-exported dropshot types in oximeter-producer configuration (#1369)
Parameterize more tests (#1364)
Misc cleanup, remove sqlite references. (#1360)
Fix `Extent::close` docstring (#1359)
Make many `Region` functions synchronous (#1356)
Remove `Workstate::Done` (unused) (#1355)
Return a sorted `VecDeque` directly (#1354)
Combine `proc_frame` and `do_work_for` (#1351)
Move `do_work_for` and `do_work` into `ActiveConnection` (#1350)
Support arbitrary Volumes during replace compare (#1349)
Remove the SQLite backend (#1352)
Add a custom timeout for buildomat tests (#1344)
Move `proc_frame` into `ActiveConnection` (#1348)
Remove `UpstairsConnection` from `DownstairsWork` (#1341)
Move Work into ConnectionState (#1340)
Make `ConnectionState` an enum type (#1339)
Parameterize `test_repair.sh` directories (#1345)
Remove `Arc<Mutex<Downstairs>>` (#1338)
Send message to Downstairs directly (#1336)
Consolidate `on_disconnected` and `remove_connection` (#1333)
Move disconnect logic to the Downstairs (#1332)
Remove invalid DTrace probes. (#1335)
Fix outdated comments (#1331)
Use message passing when a new connection starts (#1330)
Move cancellation into Downstairs, using a token to kill IO tasks (#1329)
Make the Downstairs own per-connection state (#1328)
Move remaining local state into a `struct ConnectionState` (#1327)
Consolidate negotiation + IO operations into one loop (#1322)
Allow replacement of a target in a read_only_parent (#1281)
Do all IO through IO tasks (#1321)
Make `reqwest_client` only present if it's used (#1326)
Move negotiation into Downstairs as well (#1320)
Update Rust crate clap to v4.5.4 (#1301)
Reuse a reqwest client when creating Nexus clients (#1317)
Reuse a reqwest client when creating repair client (#1324)
Add % to keep buildomat happy (#1323)
Downstairs task cleanup (#1313)
Update crutest replace test, and mismatch printing. (#1314)
Added more DTrace scripts. (#1309)
Update Rust crate async-trait to 0.1.80 (#1298)
@leftwo leftwo mentioned this issue Jun 26, 2024
Merged
leftwo added a commit that referenced this issue Jun 26, 2024
@leftwo
Update crucible, add DTrace crucible to host image (#5954)
6e29409 
Update Crucible and Propolis to the latest

Added a new package, crucible-dtrace that pulls from buildomat a package
that contains a set of DTrace scripts. These scripts are extracted into the 
global zone at /opt/oxide/crucible_dtrace/

Crucible latest includes these updates:
Clean up dependency checking, fixing space leak (#1372) Make a DTrace
package (#1367)
Use a single context in all messages (#1363)
Remove `DownstairsWork`, because it's redundant (#1371) Remove
`WorkState`, because it's implicit (#1370)
Do work immediately upon receipt of a job, if possible (#1366) Move 'do
work for one job' into a helper function (#1365) Remove `DownstairsWork`
from map when handling it (#1361) Using `block_in_place` for IO
operations (#1357)
update omicron deps; use re-exported dropshot types in oximeter-producer
configuration (#1369) Parameterize more tests (#1364)
Misc cleanup, remove sqlite references. (#1360)
Fix `Extent::close` docstring (#1359)
Make many `Region` functions synchronous (#1356)
Remove `Workstate::Done` (unused) (#1355)
Return a sorted `VecDeque` directly (#1354)
Combine `proc_frame` and `do_work_for` (#1351)
Move `do_work_for` and `do_work` into `ActiveConnection` (#1350) Support
arbitrary Volumes during replace compare (#1349) Remove the SQLite
backend (#1352)
Add a custom timeout for buildomat tests (#1344)
Move `proc_frame` into `ActiveConnection` (#1348)
Remove `UpstairsConnection` from `DownstairsWork` (#1341) Move Work into
ConnectionState (#1340)
Make `ConnectionState` an enum type (#1339)
Parameterize `test_repair.sh` directories (#1345)
Remove `Arc<Mutex<Downstairs>>` (#1338)
Send message to Downstairs directly (#1336)
Consolidate `on_disconnected` and `remove_connection` (#1333) Move
disconnect logic to the Downstairs (#1332)
Remove invalid DTrace probes. (#1335)
Fix outdated comments (#1331)
Use message passing when a new connection starts (#1330) Move
cancellation into Downstairs, using a token to kill IO tasks (#1329)
Make the Downstairs own per-connection state (#1328) Move remaining
local state into a `struct ConnectionState` (#1327) Consolidate
negotiation + IO operations into one loop (#1322) Allow replacement of a
target in a read_only_parent (#1281) Do all IO through IO tasks (#1321)
Make `reqwest_client` only present if it's used (#1326) Move negotiation
into Downstairs as well (#1320)
Update Rust crate clap to v4.5.4 (#1301)
Reuse a reqwest client when creating Nexus clients (#1317) Reuse a
reqwest client when creating repair client (#1324) Add % to keep
buildomat happy (#1323)
Downstairs task cleanup (#1313)
Update crutest replace test, and mismatch printing. (#1314) Added more
DTrace scripts. (#1309)
Update Rust crate async-trait to 0.1.80 (#1298)

Propolis just has this one update:
Allow boot order config in propolis-standalone
---------

Co-authored-by: Alan Hanson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
networking Related to the networking.
Projects
None yet
Milestone
MVP
Development

No branches or pull requests
2 participants
@luqmana @bnaecker