"binascii.Error: Incorrect padding" when receiving binary message from Swift SocketIO client

[fhfuih]

I am using python SocketIO as a server, and I will transmit images from the client to the server. The images will be binary data in PNG format. When the python server receives such messages from a Swift client, the following error presents:

post request handler error
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/site-packages/engineio/async_server.py", line 306, in handle_request
    await socket.handle_post_request(environ)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/site-packages/engineio/async_socket.py", line 109, in handle_post_request
    p = payload.Payload(encoded_payload=body)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/site-packages/engineio/payload.py", line 13, in __init__
    self.decode(encoded_payload)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/site-packages/engineio/payload.py", line 45, in decode
    self.packets = [packet.Packet(encoded_packet=encoded_packet)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/site-packages/engineio/packet.py", line 28, in __init__
    self.decode(encoded_packet)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/site-packages/engineio/packet.py", line 65, in decode
    self.data = base64.b64decode(encoded_packet[1:])
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/hksi/lib/python3.12/base64.py", line 88, in b64decode
    return binascii.a2b_base64(s, strict_mode=validate)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
binascii.Error: Incorrect padding

I think this error is triggered when the server handles this event. It automatically pops up when the message arrives at the server. Then, it automatically stops the connection.

However, I also tried forging an image data in a test python SocketIO client. This time, no error is shown.

My Swift code to generate binary PNG data (of type Data) is

Image(uiImage: myUIImageVariable).pngData()
// Also handled the case when `pngData` returns `nil`

My python code to generate binary PNG data (of type bytes) is

imageBytesIO = BytesIO()
image = Image.new("RGB", (244, 244), "#B99169")
draw = ImageDraw.Draw(image)
draw.text((10, 10), f"Time: {t}", fill="black")
image.save(imageBytesIO, format="PNG")
imageBytes = imageBytesIO.getvalue()
imageBytesIO.close()
# imageBytes is the final value

I am using python-socketio version 5.11.2, and Swift SocketIO 16.1.0. I assume they are compatible according to this post. The Swift SocketIO library mentions "version three", which I think should be the JS SocketIO version instead of the protocol version.

[miguelgrinberg]

It appears the Swift port of Socket.IO is encoding base64 binary data without standard padding. I've found a stack overflow answer that suggests that the base64 encoder in Swift does not generate padding, even though this is the standard. The base64 spec says this:

Implementations MUST include appropriate pad characters at the end of
encoded data unless the specification referring to this document
explicitly states otherwise.

The Socket.IO protocol does not explicitly indicate if padding must be used or not, but the example included in the specification does use padding.

[fhfuih]

Thanks for the prompt reply! While I also plan to report this affair to the Swift package, I's like to know if there is a way that I can patch/hack the python library so that padding is added before the message is processed? I have just had a look at the source code, and it seems hard to do so.

[miguelgrinberg]

Not sure if it will work, but a quick patch would be adding this in line 65 of engineio/packet.py:

This is the original code:

    self.data = base64.b64decode(encoded_packet[1:])

Try this instead:

    self.data = base64.b64decode(encoded_packet[1:] + b'===')

[fhfuih]

And another solution is to force WebSocket (using the .forceWebsockets(true) option while creating the SocketManager. According to the SocketIO protocol linked above, Websockets will send binary data as-is without base64 encoding.

Originally I failed to use websockets because I have not configured CORS properly. They use ws:// and wss:// protocols and I have to explicitly allow them apart from http and https.