Skip to content

Commit

Permalink
make tests pass
Browse files Browse the repository at this point in the history
This is a hack because in real use, I am pretty sure that `frames` will be either all memoryviews or all not, since they'll either be coming from a comm or from a bytestring. But in tests where we call `loads` directly, this may not be the case.
  • Loading branch information
gjoseph92 committed Jul 23, 2021
1 parent 4fc3dae commit 1869b18
Showing 1 changed file with 13 additions and 2 deletions.
15 changes: 13 additions & 2 deletions distributed/protocol/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,12 +100,23 @@ def _decode_default(obj):
if deserialize:
if "compression" in sub_header:
sub_frames = decompress(sub_header, sub_frames)

# HACK: check for memoryviews in preceding frames that share an underlying
# buffer with these sub-frames, to figure out what offset in the underlying
# buffer the sub-frames start at.
memoryview_offset = 0
if sub_frames and isinstance(sub_frames[0], memoryview):
obj = sub_frames[0].obj
for f in reversed(frames[:offset]):
if not (isinstance(f, memoryview) and f.obj is obj):
break
memoryview_offset += len(f)

return merge_and_deserialize(
sub_header,
sub_frames,
deserializers=deserializers,
memoryview_offset=sum(len(f) for f in frames[:offset]),
# ^ TODO: what if not all frames are memoryviews?
memoryview_offset=memoryview_offset,
)
else:
return Serialized(sub_header, sub_frames)
Expand Down

0 comments on commit 1869b18

Please sign in to comment.