Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object safe Protobuf #2412

Merged
merged 17 commits into from
Aug 2, 2022
Merged

Conversation

hu55a1n1
Copy link
Member

@hu55a1n1 hu55a1n1 commented Jul 16, 2022

Partially addresses: #2335

Replaces tendermint_proto::Protobuf with an object safe variant.

Note: The target branch is a long-lived branch hu55a1n1/light-client-extraction. The plan is to merge PRs (addressing #2335) such as this into hu55a1n1/light-client-extraction and ultimately merge that into master.


PR author checklist:

  • Added changelog entry, using unclog.
  • Added tests: integration (for Hermes) or unit/mock tests (for modules).
  • Linked to GitHub issue.
  • Updated code comments and documentation (e.g., docs/).

Reviewer checklist:

  • Reviewed Files changed in the GitHub PR explorer.
  • Manually tested (in case integration/unit/mock tests are absent).

@hu55a1n1 hu55a1n1 changed the base branch from master to hu55a1n1/remove-clients-any-enums-usages July 16, 2022 21:00
@hu55a1n1 hu55a1n1 force-pushed the hu55a1n1/remove-clients-any-enums-usages branch from 310d796 to ff09e88 Compare July 16, 2022 21:01
@hu55a1n1 hu55a1n1 changed the base branch from hu55a1n1/remove-clients-any-enums-usages to master July 20, 2022 15:39
@hu55a1n1 hu55a1n1 changed the base branch from master to hu55a1n1/light-client-extraction July 20, 2022 16:36
@hu55a1n1 hu55a1n1 marked this pull request as ready for review July 20, 2022 16:41
@hu55a1n1 hu55a1n1 changed the title Object safe Protobuf ADR011 impl: Object safe Protobuf Jul 20, 2022
Copy link
Contributor

@seanchen1991 seanchen1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the thoroughness of the documentation! Just have a small question on the docs for the Protobuf trait.

proto/src/protobuf/mod.rs Outdated Show resolved Hide resolved
@hu55a1n1 hu55a1n1 changed the title ADR011 impl: Object safe Protobuf Object safe Protobuf Jul 26, 2022
use alloc::boxed::Box;
use core::convert::{Into as CoreInto, TryFrom as CoreTryFrom};

mod sealed {
Copy link
Contributor

@plafer plafer Jul 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove the sealed module entirely. It basically defines two "marker traits", and "marks" all types that satisfy the constraints:

  • SealedInto<T> for U: Clone + CoreInto<T>
  • SealedTryFrom<T> for U: CoreTryFrom<T>
    These markers are then used as supertraits to our Into and TryFrom.

We see the same constraints being applied to our blanket implementations. So, the supertraits to our Into and TryFrom can simply be removed; the blanket implementation takes care of the constraints.

TLDR: if we remove the sealed module, the sealed::SealedInto<T> supertrait on Into, and the sealed::SealedTryFrom<T> supertrait on TryFrom, everything compiles and we've reduced the complexity of this module.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sealed traits signal that they are not supposed to be implemented by the user. I think it's nice to have them because we provide blanket impls for these traits.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But what does having them provide that my suggestion doesn't? I understand that they're not supposed to be implemented by the user; I just don't see why they're useful at all.

TLDR: if we remove the sealed module, the sealed::SealedInto supertrait on Into, and the sealed::SealedTryFrom supertrait on TryFrom, everything compiles and we've reduced the complexity of this module.

proto/src/protobuf/mod.rs Outdated Show resolved Hide resolved
}

pub trait Into<T: ?Sized>: sealed::SealedInto<T> {
fn into(&self) -> Box<T>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should give a different name to into(), such as into_ref(). With the current version, I noticed weird behaviors in my editor. When hovering over a call to erased::Into::into() in one of the functions of our Protobuf implementation, rust-analyzer claims that the into() being referenced is the one in core::convert::Into. I even tried to explicitly write

let raw: Box<T> = self.into();

and rust-analyzer still thinks that I'm using core::convert::into. I haven't tested other language servers.

I then played around a bit in a rust playground here. I'm not sure why it's not working; perhaps it has something to do with core::convert::Into being in the prelude?

In any case, using another name for into() avoids these headaches.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an interesting problem. I used erased::Into because that is the convention that erased_serde uses and I thought that it was a common convention, but it seems we're seeing problems because we're using it with std lib traits (in the prelude). IMHO, into_ref() would be a method that takes self by value, so a better name would perhaps be to_boxed(). WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that into_ref() suggests taking ownership. On the flip slide, for me, to_boxed() suggests boxing self (i.e. no conversion to another type), such as

fn to_boxed(&self) -> Box<Self> {
    Box::new(self.clone())
}

Brainstorming alternatives:

  • into_boxed(): still into suggests taking ownership of self
  • clone_boxed(): I don't like it, doesn't suggest a conversion either
  • clone_into_boxed(): quite wordy, but perhaps captures best the conversion we're doing?
  • as_boxed(): as_* in std take a &self, and suggest a conversion?
  • ... ?

@plafer plafer merged commit 880d031 into hu55a1n1/light-client-extraction Aug 2, 2022
@plafer plafer deleted the hu55a1n1/custom-protobuf branch August 2, 2022 15:21
@hu55a1n1 hu55a1n1 mentioned this pull request Aug 16, 2022
20 tasks
romac added a commit that referenced this pull request Aug 26, 2022
* Remove all occurences of Any* enums from light-clients (#2332)

* Make ConsensusState object-safe

* Make ClientState trait object safe

* Extract ConsensusReader

* Fix clippy errors

* Remove all Any* usages from clients

* Remove client specific errors from Ics02Error

* Extract ChannelMetaReader

* Fix Ci

* Remove unnecessary Clone supertrait from Header & Misbehaviour

* Fix bad std import

* Remove redundant Clone supertrait from ClientDef

* Rename ConsensusReader -> LightClientReader

* Add docstring for LightClientReader

* Add docstring for ChannelMetaReader

* Remove unused LightClientReader trait bound

* AnyTrait -> CoreAny

* Remove ClientReader::maybe_consensus_state()

* Rename LightClientReader -> ClientReaderLightClient and ChannelReaderLightClient

Co-authored-by: Philippe Laferriere <[email protected]>

* Object safe Protobuf (#2412)

* Define protobuf error

* Implement object safe Protobuf trait

* Add test for object safety

* Provide blanket impls for ToBoxed and TryFromIfSized

* Add blanket impl test

* Use more specific bounds

* Seal Protobuf supertraits

* Add mod erased

* Rename erased traits

* Doc comments for Protobuf

* Update code to use ibc_proto::protobuf::Protobuf

* Fix no_std check

* Address review feedback

* Rename Protobuf<T> to Protobuf<Raw>

* Remove Sealed mod

* Rename Into -> CloneInto and return T instead of Box<T>

* Remove `AnyHeader` from the modules (#2524)

* Define protobuf error

* Implement object safe Protobuf trait

* Add test for object safety

* Provide blanket impls for ToBoxed and TryFromIfSized

* Add blanket impl test

* Use more specific bounds

* Seal Protobuf supertraits

* Add mod erased

* Rename erased traits

* Doc comments for Protobuf

* Update code to use ibc_proto::protobuf::Protobuf

* Fix no_std check

* Address review feedback

* Rename Protobuf<T> to Protobuf<Raw>

* commit before dealing with Serialize

* Implement erased_serde Serialize on Header

* comment redacted

* Made `Header` trait object safe

* error

* Impl Protobuf<Any> for TmHeader

* Impl Protobuf<Any> for MockHeader

* Change Header supertrait to `Protobuf<Any>`` instead of `Protobuf<RawHeader>`

* ErasedProtobuf

* Fix encode_to_string compile error

* implement downcast_header

* Move AsAny to dynamic_typing module

* fix 1 more compile error: clone

* Make UpdateClient `PartialEq` again by making `dyn Header` PartialEq

* Add `Header::into_box`

* Use `Any` in place of `MsgUpdateAnyClient.header`

* Update relayer for MsgUpdateAnyClient `Any` header changes

* Avoid using `Header` methods in update_client handler

* Remove AnyHeader from ClientDef

* Update tests for ClientDef changes

* Remove redundant clone

* Remove AnyHeader usage from mock

* MsgUpdateClient -> RawMsgUpdateClient

* MsgUpdateAnyClient -> MsgUpdateClient

* Remove use of AnyHeader

* Move MisbehaviourEvidence to the relayer

* Remove use of AnyHeader in mock

* Remove AnyHeader from modelator

* Fix MockHeader test

* Move AnyHeader into the relayer

* use alloc String in protobuf

* fix no-std issue

* Cleanup SyntheticTmBlock impl

* Rename EqualsHeader and move to sealed module

* Remove unused Error=Error

* Remove Deref impl for SyntheticTmBlock

* Fix default `trusted_height` in `HostBlock::generate_tm_block()`

* fix test

* fix errors

Co-authored-by: hu55a1n1 <[email protected]>

* Remove `AnyMisbehaviour` from modules (#2527)

* Implement PartialEq for dyn Misbehaviour

* Add DynClone to Misbehaviour

* MsgSubmitMisbehaviour now uses Any

* Move AnyMisbehaviour to the relayer

* Header: move sealed module at the end of file

* Rename misbehavior -> misbehaviour

* Misbehaviour: mark Mock stuff cfg(test)

* Header: put Mock stuff behind cfg(test)

* Remove `AnyConsensusState` from modules  (#2528)

* Update Cargo.lock

* Fix header comment

* Implement Erased traits for ConsensusState

* Implement supertraits for concrete `ConsensusState`s

* Remove AnyConsensusState from ClientDef

* Remove check for matching ClientTypes in MsgCreateAnyClient domain type conv

* Remove AnyConsensusState from client handlers

* Fix tests compile

* Fix typo in comment

* Add ConsensusState::timestamp()

* Remove `AnyConsensusState` from readers & keepers

* Remove `AnyConsensusState` from mock context

* Fix tests

* Impl conversions for Host block to dyn ConsensusState

* Treat Tm light client as first class citizen and allow Mock impl to depend on it's data structures

* Fix no-std check

* Replace AnyConsensusState in Messages with Box<dyn ConsensusState>

* Modify `verify_upgrade_and_update_state()` to accept `Any` consensus state

* Remove redundant conversion from MockHeader to AnyConsensusState

* Remove any remnants of AnyConsensusState from tests

* Move `AnyConsensusState` to the relayer crate

* Move type url consts to the light client mods

* Extract mock::consensus_state mod

* Rename client_consensus.rs to consensus_state.rs

* Move header type url consts to the light client mods

* Minor refactoring and doc comments

* Move misbehaviour type url consts to the light client mods

* Rename Clientdef::validate_consensus_state -> initialise

* Remove `AnyClientState` from the modules (#2559)

* dyn UpgradeOptions

* Object safe ClientState

* Add PartialEq for ClientState

* Remove AnyClientState

* Update remaining handlers and relayer code

* Fix tests

* Remove AnyClientState from ICS03

* Remove AnyClientState from mock

* Fix typos

* Fix CI

* Move AnyClientState to relayer crate

* Move client state type URLs to light clients

* Remove ClientReaderLightClient

* Fix comment

* Avoid decoding expected client state on counterparty (#2569)

* Merge `Clientdef` into `Clientstate` (#2570)

* add verification functions to ClientState

* Implement tendermint clientstate

* Implemented verification functions for MockClientState

* Remove ClientDef

* MsgCreateAnyClient -> MsgCreateClient

* MsgUpgradeAnyClient -> MsgUpgradeClient

* fix test

* Cleanup fully qualified imports

* Rename TmClientState

* Remove ChannelReaderLightClient (#2572)

Co-authored-by: hu55a1n1 <[email protected]>

* Re-add Deserialize on Any enums in relayer crate where possible

* Impl From<&dyn> for AnyClientState and AnyConsensusState

* Remove redundant line

Co-authored-by: Philippe Laferriere <[email protected]>
Co-authored-by: Romain Ruetschi <[email protected]>
hu55a1n1 added a commit to hu55a1n1/hermes that referenced this pull request Sep 13, 2022
* Remove all occurences of Any* enums from light-clients (informalsystems#2332)

* Make ConsensusState object-safe

* Make ClientState trait object safe

* Extract ConsensusReader

* Fix clippy errors

* Remove all Any* usages from clients

* Remove client specific errors from Ics02Error

* Extract ChannelMetaReader

* Fix Ci

* Remove unnecessary Clone supertrait from Header & Misbehaviour

* Fix bad std import

* Remove redundant Clone supertrait from ClientDef

* Rename ConsensusReader -> LightClientReader

* Add docstring for LightClientReader

* Add docstring for ChannelMetaReader

* Remove unused LightClientReader trait bound

* AnyTrait -> CoreAny

* Remove ClientReader::maybe_consensus_state()

* Rename LightClientReader -> ClientReaderLightClient and ChannelReaderLightClient

Co-authored-by: Philippe Laferriere <[email protected]>

* Object safe Protobuf (informalsystems#2412)

* Define protobuf error

* Implement object safe Protobuf trait

* Add test for object safety

* Provide blanket impls for ToBoxed and TryFromIfSized

* Add blanket impl test

* Use more specific bounds

* Seal Protobuf supertraits

* Add mod erased

* Rename erased traits

* Doc comments for Protobuf

* Update code to use ibc_proto::protobuf::Protobuf

* Fix no_std check

* Address review feedback

* Rename Protobuf<T> to Protobuf<Raw>

* Remove Sealed mod

* Rename Into -> CloneInto and return T instead of Box<T>

* Remove `AnyHeader` from the modules (informalsystems#2524)

* Define protobuf error

* Implement object safe Protobuf trait

* Add test for object safety

* Provide blanket impls for ToBoxed and TryFromIfSized

* Add blanket impl test

* Use more specific bounds

* Seal Protobuf supertraits

* Add mod erased

* Rename erased traits

* Doc comments for Protobuf

* Update code to use ibc_proto::protobuf::Protobuf

* Fix no_std check

* Address review feedback

* Rename Protobuf<T> to Protobuf<Raw>

* commit before dealing with Serialize

* Implement erased_serde Serialize on Header

* comment redacted

* Made `Header` trait object safe

* error

* Impl Protobuf<Any> for TmHeader

* Impl Protobuf<Any> for MockHeader

* Change Header supertrait to `Protobuf<Any>`` instead of `Protobuf<RawHeader>`

* ErasedProtobuf

* Fix encode_to_string compile error

* implement downcast_header

* Move AsAny to dynamic_typing module

* fix 1 more compile error: clone

* Make UpdateClient `PartialEq` again by making `dyn Header` PartialEq

* Add `Header::into_box`

* Use `Any` in place of `MsgUpdateAnyClient.header`

* Update relayer for MsgUpdateAnyClient `Any` header changes

* Avoid using `Header` methods in update_client handler

* Remove AnyHeader from ClientDef

* Update tests for ClientDef changes

* Remove redundant clone

* Remove AnyHeader usage from mock

* MsgUpdateClient -> RawMsgUpdateClient

* MsgUpdateAnyClient -> MsgUpdateClient

* Remove use of AnyHeader

* Move MisbehaviourEvidence to the relayer

* Remove use of AnyHeader in mock

* Remove AnyHeader from modelator

* Fix MockHeader test

* Move AnyHeader into the relayer

* use alloc String in protobuf

* fix no-std issue

* Cleanup SyntheticTmBlock impl

* Rename EqualsHeader and move to sealed module

* Remove unused Error=Error

* Remove Deref impl for SyntheticTmBlock

* Fix default `trusted_height` in `HostBlock::generate_tm_block()`

* fix test

* fix errors

Co-authored-by: hu55a1n1 <[email protected]>

* Remove `AnyMisbehaviour` from modules (informalsystems#2527)

* Implement PartialEq for dyn Misbehaviour

* Add DynClone to Misbehaviour

* MsgSubmitMisbehaviour now uses Any

* Move AnyMisbehaviour to the relayer

* Header: move sealed module at the end of file

* Rename misbehavior -> misbehaviour

* Misbehaviour: mark Mock stuff cfg(test)

* Header: put Mock stuff behind cfg(test)

* Remove `AnyConsensusState` from modules  (informalsystems#2528)

* Update Cargo.lock

* Fix header comment

* Implement Erased traits for ConsensusState

* Implement supertraits for concrete `ConsensusState`s

* Remove AnyConsensusState from ClientDef

* Remove check for matching ClientTypes in MsgCreateAnyClient domain type conv

* Remove AnyConsensusState from client handlers

* Fix tests compile

* Fix typo in comment

* Add ConsensusState::timestamp()

* Remove `AnyConsensusState` from readers & keepers

* Remove `AnyConsensusState` from mock context

* Fix tests

* Impl conversions for Host block to dyn ConsensusState

* Treat Tm light client as first class citizen and allow Mock impl to depend on it's data structures

* Fix no-std check

* Replace AnyConsensusState in Messages with Box<dyn ConsensusState>

* Modify `verify_upgrade_and_update_state()` to accept `Any` consensus state

* Remove redundant conversion from MockHeader to AnyConsensusState

* Remove any remnants of AnyConsensusState from tests

* Move `AnyConsensusState` to the relayer crate

* Move type url consts to the light client mods

* Extract mock::consensus_state mod

* Rename client_consensus.rs to consensus_state.rs

* Move header type url consts to the light client mods

* Minor refactoring and doc comments

* Move misbehaviour type url consts to the light client mods

* Rename Clientdef::validate_consensus_state -> initialise

* Remove `AnyClientState` from the modules (informalsystems#2559)

* dyn UpgradeOptions

* Object safe ClientState

* Add PartialEq for ClientState

* Remove AnyClientState

* Update remaining handlers and relayer code

* Fix tests

* Remove AnyClientState from ICS03

* Remove AnyClientState from mock

* Fix typos

* Fix CI

* Move AnyClientState to relayer crate

* Move client state type URLs to light clients

* Remove ClientReaderLightClient

* Fix comment

* Avoid decoding expected client state on counterparty (informalsystems#2569)

* Merge `Clientdef` into `Clientstate` (informalsystems#2570)

* add verification functions to ClientState

* Implement tendermint clientstate

* Implemented verification functions for MockClientState

* Remove ClientDef

* MsgCreateAnyClient -> MsgCreateClient

* MsgUpgradeAnyClient -> MsgUpgradeClient

* fix test

* Cleanup fully qualified imports

* Rename TmClientState

* Remove ChannelReaderLightClient (informalsystems#2572)

Co-authored-by: hu55a1n1 <[email protected]>

* Re-add Deserialize on Any enums in relayer crate where possible

* Impl From<&dyn> for AnyClientState and AnyConsensusState

* Remove redundant line

Co-authored-by: Philippe Laferriere <[email protected]>
Co-authored-by: Romain Ruetschi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants