Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move generated protos into oak_abi #834

Merged
merged 3 commits into from
Apr 9, 2020
Merged

Move generated protos into oak_abi #834

merged 3 commits into from
Apr 9, 2020

Conversation

daviddrysdale
Copy link
Contributor

@daviddrysdale daviddrysdale commented Apr 9, 2020

Protocol buffer messages that are exchanged in serialized form between
Wasm Nodes and Oak-provided pseudo-Nodes are effectively part of the
Oak ABI, so move the generated code for them into the oak_abi crate.

This allows us to remove the oak_runtime->oak dependency that was
introduced by #772.

Along the way, put the gRPC encapsulation protobuf messages into an
encap submodule/namespace.

Fixes #764.

Checklist

  • Pull request affects core Oak functionality (e.g. runtime, SDK, ABI)
    • I have written tests that cover the code changes.
    • I have checked that these tests are run by Cloudbuild
    • I have updated documentation accordingly.
    • I have raised an issue to
      cover any TODOs and/or unfinished work.
  • Pull request includes prototype/experimental work that is under
    construction.

@daviddrysdale daviddrysdale added the WIP Work in progress label Apr 9, 2020
@daviddrysdale daviddrysdale marked this pull request as ready for review April 9, 2020 14:11
@daviddrysdale daviddrysdale removed the WIP Work in progress label Apr 9, 2020
@daviddrysdale daviddrysdale requested a review from tiziano88 April 9, 2020 14:15
@daviddrysdale
Copy link
Contributor Author

I'm in two minds as to whether to keep the last two commits. On the one hand, it feels more complete to include the storage channel messages in oak_abi, as they form part of the ABI. On the other hand, the only code to use them (the oak::storage SDK code) needs its own copy anyway (because it needs the full service definition, which can't be included in oak_abi because it assumes the use of the oak SDK crate).

pub mod oak {
use oak_abi::proto::oak::label;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment to clarify why this re-export is needed? Presumably we would like to make it so that developers only need to depend on oak and not oak_abi directly? I guess that makes sense, though I'm always a bit hesitant to introduce re-exports, as having multiple names for the same things can be confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment – the storage message definitions rely on the Label message, which is now (only) generated under the oak_abi crate.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, may as well leave them out of the ABI then I guess, there is not much point in having them there. I don't really mind either way though.


message StorageChannelRollbackResponse {
}
import "oak/proto/storage_messages.proto";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow what's going on here. Why is the service definition not part of the ABI? It seems it should be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated code for the service definition assumes the use of the Oak SDK code, and so cannot live in the oak_abi crate. That leads to the separation of the message definitions (which only depend on prost and so can live in oak_abi) from the service definition generated code (still in oak, but now with a parallel copy of the message definitions), which leads to my earlier question – is it worth bothering with the storage protos?

@@ -16,73 +16,9 @@

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps this file should be renamed to storage_node or storage_service at some point? (Or just deleted -- hint hint)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable, but for another day.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, done: it's now StorageService in oak/proto/storage_service.proto

Protocol buffer messages that are exchanged in serialized form between
Wasm Nodes and Oak-provided pseudo-Nodes are effectively part of the
Oak ABI, so move the generated code for them into the oak_abi crate.

This allows us to remove the oak_runtime->oak dependency that was
introduced by #772.

Along the way, put the gRPC encapsulation protobuf messages into an
`encap` submodule/namespace.

Fixes #764.
The storage.proto file holds the gRPC service definition for the
external storage processor; this is only needed by the C++ storage
pseudo-Node code.
@daviddrysdale daviddrysdale requested a review from tiziano88 April 9, 2020 14:42
@daviddrysdale
Copy link
Contributor Author

Dropped the last two commits (which split the storage proto and copied the message definitions to oak_abi) and added a rename commit for storage_channel.proto.

Also move into a submodule/namespace oak.storage, and rename the
StorageNode service to StorageService.
@daviddrysdale daviddrysdale merged commit 0bb3b55 into project-oak:master Apr 9, 2020
@daviddrysdale daviddrysdale deleted the abi-proto branch April 9, 2020 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move ABI protobuf code to oak_abi crate
3 participants