Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: refactor bounded vec to zero copy #1444

Merged
merged 4 commits into from
Jan 5, 2025

Conversation

ananas-block
Copy link
Contributor

@ananas-block ananas-block commented Jan 2, 2025

Issue:

  1. bounded vec api exposes more unsafe functions than necessary and its api is not easy to use
  2. batched-merkle-tree needed to wrap bounded vecs into manually drop

Solution:

  1. refactor light-bounded-vec so that unsafe operations are unified and wrapped
  2. remove light-bounded-vec and rename refactored crate to light-zero-copy
  3. in light-batched-merkle-tree replace all instances of bound vec with zero copy vec, zero copy slice or zero copy cyclic vec
  4. remove zero copy helpers from light-batched-merkle-tree

Changes:

  1. light-batched-merkle-tree
    1. remove ManuallyDrop
    2. batched queue
      1. replace *mut BatchedQueueMetadata -> WrappedPointerMut<BatchedQueueMetadata>
      2. replace ManuallyDrop<BoundedVec<Batch>> -> ZeroCopySliceMutUsize<Batch>
      3. replace Vec<ManuallyDrop<BoundedVec<[u8; 32]>>> -> Vec<ZeroCopyVecUsize<[u8; 32]>>
      4. replace Vec<ManuallyDrop<BoundedVec<u8>>> -> Vec<ZeroCopySliceMutUsize<u8>>
      5. replace Vec<ManuallyDrop<BoundedVec<[u8; 32]>>> -> Vec<ZeroCopyVecUsize<[u8; 32]>>
      6. deref & deref_mut metadata -> can remove get_metadata and get_metadata_mut in a different pr
    3. batched merkle tree
      1. replace *mut BatchedMerkleTreeMetadata -> WrappedPointerMut<BatchedMerkleTreeMetadata>
      2. replace ManuallyDrop<CyclicBoundedVec<[u8; 32]>> -> ZeroCopyCyclicVecUsize<[u8; 32]>
      3. replace ManuallyDrop<BoundedVec<Batch>> -> ZeroCopySliceMutUsize<Batch>
      4. replace Vec<ManuallyDrop<BoundedVec<[u8; 32]>>> -> Vec<ZeroCopyVecUsize<[u8; 32]>>
      5. replace Vec<ManuallyDrop<BoundedVec<u8>>> -> Vec<ZeroCopySliceMutUsize<u8>>
      6. replace Vec<ManuallyDrop<BoundedVec<[u8; 32]>>> -> Vec<ZeroCopyVecUsize<[u8; 32]>>
      7. deref & deref_mut metadata -> can remove get_metadata and get_metadata_mut in a different pr
  2. light-zero-copy
    1. WrappedPointer: wrapper around immutable raw pointer
    2. WrappedPointerMut: wrapper around mutable raw pointer
    3. ZeroCopySliceMut:
      1. wraps a length WrappedPointer and data MaybeUninit<*mut T>
      2. use case are fixed length zero copy slices which are initialized with full length (if we need to push we take a zero copy vec)
      3. wraps slice
    4. ZeroCopyVec
      1. wraps ZeroCopySliceMut , treats slice length as capacity and adds dynamic length as WrappedPointerMut
      2. supports pushing values up to capacity
    5. ZeroCopyCyclicVec
      1. wraps ZeroCopyVec , adds current_index as WrappedPointerMut and adds cyclic functionality

@ananas-block ananas-block force-pushed the jorrit/feat-refactor-bounded-vec-to-zero-copy branch 4 times, most recently from 1ffbafb to f0153d8 Compare January 2, 2025 23:02
Comment on lines +285 to +290
// TODO: remove
pub fn get_metadata(&self) -> &BatchedMerkleTreeMetadata {
unsafe { self.metadata.as_ref() }.unwrap()
self.metadata.get()
}
// TODO: remove
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in different pr

Comment on lines +173 to +179
// TODO: remove
pub fn get_metadata(&self) -> &BatchedQueueMetadata {
unsafe { &*self.metadata }
self.metadata.get()
}

// TODO: remove
pub fn get_metadata_mut(&mut self) -> &mut BatchedQueueMetadata {
unsafe { &mut *self.metadata }
self.metadata.get_mut()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in different pr

@ananas-block ananas-block force-pushed the jorrit/feat-refactor-bounded-vec-to-zero-copy branch 2 times, most recently from 39463dc to 668ddcd Compare January 3, 2025 16:16
@ananas-block ananas-block force-pushed the jorrit/feat-refactor-bounded-vec-to-zero-copy branch from 10d88fc to 95e9bf0 Compare January 4, 2025 21:14
@sergeytimoshin sergeytimoshin self-requested a review January 4, 2025 22:37
@ananas-block ananas-block force-pushed the jorrit/feat-refactor-bounded-vec-to-zero-copy branch from 0ed92f0 to ef0c4c5 Compare January 4, 2025 23:44
@ananas-block ananas-block merged commit 4cbc426 into main Jan 5, 2025
16 checks passed
@ananas-block ananas-block deleted the jorrit/feat-refactor-bounded-vec-to-zero-copy branch January 5, 2025 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants