Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(swap): Upgrade libp2p to 0.53.2 #109

Draft
wants to merge 86 commits into
base: master
Choose a base branch
from
Draft

Conversation

binarybaron
Copy link

@binarybaron binarybaron commented Oct 10, 2024

Our libp2p version is out of date, and we need to tackle the upgrade even though it's a significant undertaking. This'll also fix some other issues.

@binarybaron binarybaron changed the title Upgrade libp2p to 0.53.2 refactor(swap): Upgrade libp2p to 0.53.2 Oct 11, 2024
- Update libp2p dependency to 0.53.2 in Cargo.toml
- Add libp2p-swarm-derive dependency
- Update imports and usage of libp2p types and traits
- Refactor NetworkBehaviour implementations to match new API
- Update Identify and Ping behavior configurations
- Adjust event handling in network behaviors
- Fix compiler errors and warnings related to API changes
…nal cbor impl, remove custom one (See: Remove custom cbor request response implementation, replaced with one from library (See: github.com/libp2p/rust-libp2p/pull/3952)
@binarybaron
Copy link
Author

binarybaron commented Oct 18, 2024

Here's an updated TODO list generated by Claude. Again, mostly for getting an overview over what needs to be done. Each issue needs to be looked into individually of course.

Claude Compiler Summary

Unresolved Imports

  • Resolve TokioDnsConfig import in swap/src/cli/transport.rs. This type might be from an older version of libp2p.
  • Resolve TokioTcpConfig import in swap/src/cli/transport.rs. This type might be from an older version of libp2p.
  • Resolve ConnectionId import in swap/src/network/redial.rs. This type might have been moved or renamed in a newer version of libp2p.
  • Resolve protocols_handler import in swap/src/network/redial.rs. This module might have been restructured in libp2p.
  • Resolve NetworkBehaviourAction and PollParameters imports in swap/src/network/redial.rs. These types might have been moved or renamed.
  • Resolve NegotiatedSubstream import in swap/src/network/swap_setup.rs. This type might have been moved or renamed.
  • Resolve ConnectionId import in swap/src/network/swap_setup/alice.rs. This type might have been moved or renamed.
  • Resolve multiple imports in swap/src/network/swap_setup/alice.rs. These types might have been moved or renamed in libp2p.
  • Resolve from_fn and FromFnUpgrade imports in swap/src/network/swap_setup.rs. These might be from an older version of libp2p.
  • Resolve SwarmBuilder import in swap/src/network/swarm.rs. This type might have been moved or renamed.
  • Resolve tokio import in swap/src/network/tor_transport.rs. The tokio module structure in libp2p might have changed.
  • Resolve mplex import in swap/src/network/transport.rs. This module might have been restructured in libp2p.
  • Resolve NoiseConfig and X25519Spec imports in swap/src/network/transport.rs. These types might have been moved or renamed.

Missing Trait Members

  • Implement missing ProtocolsHandler and OutEvent types for NetworkBehaviour in swap/src/network/redial.rs. The NetworkBehaviour trait might have been updated with new associated types.
  • Implement missing methods for NetworkBehaviour in swap/src/network/redial.rs. New methods might have been added to the NetworkBehaviour trait.
  • Implement missing ProtocolsHandler and OutEvent types for NetworkBehaviour in swap/src/network/swap_setup/alice.rs. The NetworkBehaviour trait might have been updated with new associated types.
  • Implement missing methods for NetworkBehaviour in swap/src/network/swap_setup/alice.rs. New methods might have been added to the NetworkBehaviour trait.

Type Mismatches

  • Fix type mismatch in swap/src/asb/event_loop.rs:204. Expected &OutboundRequestId, found &InboundRequestId.
  • Fix type mismatch in swap/src/cli/event_loop.rs:232. Expected InboundRequestId, found OutboundRequestId.
  • Fix type mismatch in swap/src/cli/event_loop.rs:245. Expected InboundRequestId, found OutboundRequestId.
  • Fix type mismatch in swap/src/cli/event_loop.rs:258. Expected InboundRequestId, found OutboundRequestId.
  • Fix type mismatch in swap/src/cli/list_sellers.rs:219. Expected PeerId, found Multihash<64>.

Other Errors

  • Fix pattern matching in swap/src/asb/event_loop.rs:313. The RegisterFailed variant is not a tuple struct or tuple variant.
  • Fix missing field in pattern in swap/src/cli/list_sellers.rs:176. Add the connection_id field to the pattern.
  • Resolve ambiguous associated types in swap/src/network/swap_setup/alice.rs. The associated types are not clearly defined for the implementing type.
  • Fix method signature mismatch for poll in swap/src/network/redial.rs:96. The method has 1 type parameter but the trait declaration has 0.
  • Fix method signature mismatch for poll in swap/src/network/swap_setup/alice.rs:194. The method has 1 type parameter but the trait declaration has 0.
  • Implement missing trait items for Transport in swap/src/network/tor_transport.rs:25. The Transport trait might have been updated with new methods.
  • Fix method signature mismatch for listen_on in swap/src/network/tor_transport.rs:32. The method has 2 parameters but the declaration in the trait has 3.

@binarybaron
Copy link
Author

@Einliterflasche I think this is reasonably distributed across the code base that it'll be possible to work on this concurrently. Feel free to push to this branch.

@binarybaron
Copy link
Author

binarybaron commented Oct 21, 2024

Todo List for Compiler Errors

  • Resolve import issues:

    • NegotiatedSubstream from libp2p::swarm in swap/src/network/swap_setup.rs
    • ConnectionId from libp2p::core::connection in swap/src/network/swap_setup/alice.rs
    • Various imports from libp2p::swarm in swap/src/network/swap_setup/alice.rs
    • from_fn and FromFnUpgrade from libp2p::core::upgrade in swap/src/network/swap_setup.rs
    • SwarmBuilder from libp2p::swarm in swap/src/network/swarm.rs
    • mplex and related types from libp2p in swap/src/network/transport.rs
    • NoiseConfig and X25519Spec from libp2p::noise in swap/src/network/transport.rs
  • Update NetworkBehaviour trait implementation in swap/src/network/swap_setup/alice.rs:

    • Remove or update ProtocolsHandler associated type
    • Remove or update OutEvent associated type
    • Remove or update methods: new_handler, addresses_of_peer, inject_connected, inject_disconnected, inject_event
  • Fix mplex crate usage in swap/src/network/transport.rs:

    • Resolve the undeclared crate or module mplex
  • Update upgrade module function calls in swap/src/network/swap_setup.rs:

    • Replace upgrade::read_length_prefixed with an alternative
    • Replace upgrade::write_length_prefixed with an alternative
  • Resolve type mismatches:

    • Fix mismatched types for InboundRequestId and OutboundRequestId in swap/src/asb/event_loop.rs and swap/src/cli/event_loop.rs
    • Correct PeerId and Multihash<64> type mismatch in swap/src/cli/list_sellers.rs
  • Update poll method signature in swap/src/network/swap_setup/alice.rs:

    • Align the number of type parameters with the trait declaration
  • Implement missing trait items for NetworkBehaviour in swap/src/network/swap_setup/alice.rs:

    • Add ConnectionHandler associated type
    • Add ToSwarm associated type
    • Implement handle_established_inbound_connection, handle_established_outbound_connection, on_swarm_event, and on_connection_handler_event methods

See: libp2p/rust-libp2p#3920

This should be backwards compatible because SelectUpgrade should allow older clients to use yamux instead of mplex
@binarybaron
Copy link
Author

binarybaron commented Oct 21, 2024

* [ ]  Update `upgrade` module function calls in `swap/src/network/swap_setup.rs`:
  
  * [ ]  Replace `upgrade::read_length_prefixed` with an alternative
  * [ ]  Replace `upgrade::write_length_prefixed` with an alternative

These functions were removed in this PR. I haven't been able to figure out quite what to replace our current approach with. Probably some combination of asynchronous-codec, quick-protobuf-codec and unsigned-varint.

We should use libp2p/rust-libp2p#4787 as a starting point on how to migrate.

@binarybaron
Copy link
Author

@Einliterflasche Do you want to work on swap/src/network/swap_setup/alice.rs?

I'll work on what I described in the comment above.

@Einliterflasche
Copy link

I partially implemented swap/src/network/swap_setup/alice.rs, I'll have to check if I moved the function bodies correctly. I also fixed some of the import issues. It's down to 27 errors now, though there are two things I don't know how to fix: KeepAlive in network\swap_setup\alice.rs doesn't seem to be exported anywhere, and I commented out NetworkBehaviourEventProcess as well


/// Tracks [`transfer_proof::Request`]s which could not yet be sent because
/// we are currently disconnected from the peer.
buffered_transfer_proofs: HashMap<PeerId, Vec<(transfer_proof::Request, bmrng::Responder<()>)>>,
///
/// If we are not connected to the peer, the transfer proof is essentially moved from [`send_transfer_proof`] to here.
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved from outgoing_transfer_proof_queue (renamed)

@Einliterflasche
Copy link

Nit: in cli/event_loop.rs we could unnest the error handling introduced in 3e457ed to

match result {
    Ok(..) => ...,
    Err(bmrng::Error::TimeOutError) => ...,
    Err(bmrng::Error::RecvError) => ...,
}

Beyond that, I'm very excited we're cleaning up the network stack ;)

@binarybaron
Copy link
Author

binarybaron commented Oct 31, 2024

These bmrng channels are driving me insane.

thread 'tokio-runtime-worker' panicked at /Users/***/Documents/GitHub/unstoppableswap-gui-rs/swap/src/asb/event_loop.rs:673:29:
internal error: entered unreachable code: We construct the channel without a timeout, so this should never happen

@Einliterflasche
Copy link

Einliterflasche commented Oct 31, 2024

Also, the crate hasn't been updated in 3 years. We could switch to a more maintained channel implementation and implement the (simple (the original crate is <100 lines)) request-response mechanism ourselves

@binarybaron
Copy link
Author

Also, the crate hasn't been updated in 3 years. We could switch to a more maintained channel implementation and implement the (simple (the original crate is <100 lines)) request-response mechanism ourselves

To be fair: It's a pretty simple crate. Not a lot to maintain.

…perative xmr redeem requests. This is now handled by propagating network timeouts
…e tokio task which was used to forward transfer proof from one channel to another, send transfer proof directly to event loop channel
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants