Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialization: zero-copy merge subframes when possible #5208

Merged

Commits on Aug 12, 2021

  1. Zero-copy merge subframes with pointer arithmetic

    Use NumPy to get the pointers of the memoryviews and do pointer arithmetic. This is way more reliable and self-contained.
    
    TODO unit test for `merge_memoryviews` and deal with NumPy situation
    gjoseph92 committed Aug 12, 2021
    Configuration menu
    Copy the full SHA
    0f90ee7 View commit details
    Browse the repository at this point in the history
  2. reorg and handle NumPy

    gjoseph92 committed Aug 12, 2021
    Configuration menu
    Copy the full SHA
    c463a45 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a2f7243 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cc3999d View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    78ca265 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2021

  1. Fall back to NumPy for read-only buffers

    It's very annoying: `from_buffer` requires the buffer to be writeable, because ctypes is inconsistently strict about const correctness (see https://bugs.python.org/issue11427#msg148586). I don't think there's any clever pointer-casting we could do as mentioned in that thread, since we need to access through the buffer protocol—we can't just cast `mv.obj` to `c_void_p`, since we need the pointer into the current location within the buffer, not just at the start of the buffer.
    
    So for read-only buffers, we try to use NumPy, and if it's not available, we'll end up silently(!) falling back on copying.
    gjoseph92 committed Aug 13, 2021
    Configuration menu
    Copy the full SHA
    ac3eaff View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cc9cee3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    22f246d View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2021

  1. Configuration menu
    Copy the full SHA
    439be17 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0032822 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3315441 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2021

  1. Don't disable exception chaining

    Co-authored-by: crusaderky <[email protected]>
    gjoseph92 and crusaderky authored Aug 17, 2021
    Configuration menu
    Copy the full SHA
    da8ee57 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2021

  1. Update distributed/protocol/utils.py

    Co-authored-by: jakirkham <[email protected]>
    gjoseph92 and jakirkham authored Oct 5, 2021
    Configuration menu
    Copy the full SHA
    82ec2ca View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    edaeb7d View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2021

  1. Configuration menu
    Copy the full SHA
    ccea5c4 View commit details
    Browse the repository at this point in the history
  2. Fix formatting

    fjetter committed Oct 7, 2021
    Configuration menu
    Copy the full SHA
    0d1e6d1 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2021

  1. more ci

    gjoseph92 committed Oct 8, 2021
    Configuration menu
    Copy the full SHA
    2e0ae4e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    269ef1f View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2021

  1. Configuration menu
    Copy the full SHA
    6574588 View commit details
    Browse the repository at this point in the history
  2. mypy

    gjoseph92 committed Nov 1, 2021
    Configuration menu
    Copy the full SHA
    f141867 View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2021

  1. Configuration menu
    Copy the full SHA
    69e5cf2 View commit details
    Browse the repository at this point in the history