asyncio proactor udp transport stops responding after send to port that isn't listening

[esoma]

BPO	47071
Nosy	@pfmoore, @tjguk, @asvetlov, @zware, @1st1, @zooba, @esoma
PRs	gh-91227: Ignore ERROR_PORT_UNREACHABLE #32011
Files	udpbug2.py windows_events.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-03-20.14:58:17.671>
labels = ['3.10', 'OS-windows', 'expert-asyncio']
title = "asyncio proactor udp transport stops responding after send to port that isn't listening"
updated_at = <Date 2022-03-20.15:48:54.679>
user = 'https://github.com/esoma'

bugs.python.org fields:

activity = <Date 2022-03-20.15:48:54.679>
actor = 'esoma'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Windows', 'asyncio']
creation = <Date 2022-03-20.14:58:17.671>
creator = 'esoma'
dependencies = []
files = ['50691', '50692']
hgrepos = []
issue_num = 47071
keywords = ['patch']
message_count = 4.0
messages = ['415611', '415612', '415614', '415617']
nosy_count = 7.0
nosy_names = ['paul.moore', 'tim.golden', 'asvetlov', 'zach.ware', 'yselivanov', 'steve.dower', 'esoma']
pr_nums = ['32011']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue47071'
versions = ['Python 3.10']

Linked PRs

• [3.12] gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (GH-32011) #117209
• [3.11] gh-91227: Ignore ERROR_PORT_UNREACHABLE in proactor recvfrom() (GH-32011) #117210

[esoma]

Reproducer attached. Change USE_PROACTOR to False to use the SelectorEventLoop instead, which doesn't exhibit this behavior.

The output on my machine when using the proactor loop is:

datagram received b'ping 1'
datagram received b'ping 2'

And the selector loop (which is the behavior I would expect):

datagram received b'ping 1'
datagram received b'ping 2'
datagram received b'ping 3'

At a high level, after sending data to an address that isn't listening the asyncio protocol will no longer receive messages.

Digging deeper, _ProactorDatagramTransport._loop_reading encounters the windows error 1234 (ERROR_PORT_UNREACHABLE) after the "bad send". It appears this a (undocumented/buggy?) behavior of WSARecvFrom where the next call to it after an unreachable WSASendTo in UDP mode will return the ICMP unreachable message. The actual error is returned from GetOverlappedResult.

I've hacked together a fix that retries IocpProactor.recvfrom if the result is ERROR_PORT_UNREACHABLE. It fixes the issue for the reproducer and my actual use case, but it's probably not ideal. My solution for the moment is just to use the SelectorEventLoop instead.

[esoma]

Uploading my hack to asyncio.windows_events.py -- this is based off 3.10.2's distribution.

[asvetlov]

Thanks for the report.
Could you make a pull request?
The whole attached windows_events.py is hard to analyze.

[esoma]

Certainly: #32011

[msoxzw]

As far as I know, there are several alternative solutions from open source projects:
The Go programming language - https://github.com/golang/go/blob/ceefd3a37bb2a77b1e9a76fc74360c68ef16ddb8/src/internal/poll/fd_windows.go#L341-L352
The libuv - https://github.com/libuv/libuv/blob/6c692ad1cbcc5083ec90954a4b091b660aedfc10/src/win/udp.c#L462-L486

[esoma]

Looks like the libuv solution is similar to my PR -- just trying again.

The go solution is much simpler though. I'll try that out.

[msoxzw]

The libuv ignores WSAECONNRESET at lower level, so it is unnecessary to try again.

[esoma]

Ah, yeah I see. iirc, that was something I tried initially, but I was concerned about whether that would have impact elsewhere since Overlapped.getresult is also used by the connection based protocols where ERROR_PORT_UNREACHABLE probably should not be ignored. I think I was also getting some undesirable behavior, but I can't reproduce it so maybe I was just doing something weird. I suppose if it is an issue that can be taken care of with some internal state to the Overlapped object that tracks whether the last WSA command was WSARecvFrom or not and ignoring conditionally based on that. For the record, I've tested and ignoring it fixes the reproducer.

I thought about the Go approach a bit more I'm not sure where the appropriate place for this would even be. It seems inappropriate to change this behavior in the socket extension or for _overlapped to silently change this aspect an externally created socket.

I've also went back and tried just ignoring the error in window_events.IocpProactor.recvfrom.<locals>.finish_recv, since we can specifically target WSARecvFrom results that way, and it works just fine. I'm not sure why I thought it needed to be so complicated. There already exists a convention for returning no data to indicate "ignore this and keep going". I'll update my PR to use this method.

[msansoni]

Looks duplicate to #88906

[jaycelq]

Any update on this issue?