AssertionError: Real message size 4 != expected size 2228

[MatthewScholefield]

Only when sending multiple messages, I relatively consistently get this:

There was an unhandled exception with peer id b'8tz5...'. This peer is being disconnected, and the relevant exception is added to the debug queue. If you'd like to report this, please post a copy of your MeshSocket.status to git.p2p.today/issues.
Traceback (most recent call last):
  File "py2p/base.py", line 314, in process_data
    handler.found_terminator()
  File "py2p/mesh.py", line 60, in found_terminator
    msg = super(MeshConnection, self).found_terminator()
  File "py2p/base.py", line 197, in found_terminator
    msg = InternalMessage.feed_string(raw_msg, False, self.compression)
  File "py2p/messages.py", line 239, in feed_string
    _string = cls.__sanitize_string(string, sizeless)
  File "py2p/messages.py", line 174, in __sanitize_string
    len(_string), unpack_value(_string[:4]) + 4, _string))
AssertionError: Real message size 4 != expected size 2228. Buffer given: b'\x00\x00\x08\xb0'

My guess is that somehow bytes of one message are spliced in with bytes of another when two writes happen at the same time.

Note: using a lock around all calls to MeshSocket.send does not fix the issue. However, doing a sock.connect right after immediately reconnects and fixes it.

[LivInTheLookingGlass]

A lock on sock.send() would not fix this. This is happening because it is reading from the socket incorrectly.

Normal flow is something like this:

1. Check select for any ready sockets
2. On each ready socket, grab data and throw it into the buffer for that connection.
3. If the first 4 bytes indicate that you have enough data, discard those bytes, and extract the amount they indicated from the head of the buffer
4. If more data is in the buffer, repeat step 3

It seems that whats happening here is that either the mesh socket reads the header wrong, there is a race condition (shouldn't happen, but possible), or that it is sent an incorrect header.

I have those in roughly the order I think is likely. I think that the buffer modifications that I do are atomic, but its possible that the interpreter switches away mid-assignment, I suppose.

[LivInTheLookingGlass]

I would be very curious to see what happens if you were to put a lock on the connection buffer. I give it a 1 in 4 chance that it eliminates the issue.