Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEFAULT_RCVBUF can be (again) small with more than 127 VFs #1044

Closed
elajkat opened this issue Oct 18, 2022 · 0 comments · Fixed by #1045
Closed

DEFAULT_RCVBUF can be (again) small with more than 127 VFs #1044

elajkat opened this issue Oct 18, 2022 · 0 comments · Fixed by #1045

Comments

@elajkat
Copy link
Contributor

elajkat commented Oct 18, 2022

In our lab we had NICs with more and more VFs with recent increase of DEFAULT_RCVBUF to 65536, we can have more than 127 VFs. The previous attempts to push out this limit (#813 or #751 ) just pushed out the issue but not solved it.

One of our experts checked the problem, and suggested a solution to detect the real buffer size for receiving netlink, something like this in NetlinkSocketBase:
from socket import MSG_DONTWAIT, MSG_PEEK, MSG_TRUNC
def peek_bufsize(socket_descriptor):
data = bytearray()
bufsize, _ = socket_descriptor.recvfrom_into(data, 0, MSG_DONTWAIT | MSG_PEEK | MSG_TRUNC)
return bufsize

def recv(self, *argv, **kwarg):
....
return self._sock.recv(self._peek_bufsize(self._sock), **kwarg)
# return self._sock.recv(*argv, **kwarg)

This can be easily check in a dummy environment (I used a VM without sriov or any extra HW):
import sys
from pyroute2 import IPRoute

device_idx = int(sys.argv[1])
with IPRoute() as ip:
print(ip.link('get', index=device_idx, ext_mask=1))

And changing DEFAULT_RCVBUF to 1024.
I got NLMSG_ERROR with the original code, and with peek_bufsize I just got the listing of links as I expected.

elajkat added a commit to elajkat/pyroute2 that referenced this issue Oct 18, 2022
NetlinkSocketBase.recv uses DEFAULT_RCVBUF to fetch things like ip links
via netlink sockets. It happed in the last cycles that with new NICs
with higher VF numbers or similar we have to change the buffer size
(see svinota#751 or svinota#813).
A better solution (thanks to one or our experts) can be to check the
necessary buffer size before receiving the sockets.

Fixes svinota#1044

Change-Id: I87c75e4d424653e5a29408b3ac2ba8504cb2db49
elajkat added a commit to elajkat/pyroute2 that referenced this issue Oct 19, 2022
NetlinkSocketBase.recv uses DEFAULT_RCVBUF to fetch things like ip links
via netlink sockets. It happed in the last cycles that with new NICs
with higher VF numbers or similar we have to change the buffer size
(see svinota#751 or svinota#813).
A better solution (thanks to one or our experts) can be to check the
necessary buffer size before receiving the sockets.

Fixes svinota#1044

Change-Id: Ide711f27c99e4dfb75fb579f10f005cb8e1b9b37
svinota added a commit that referenced this issue Oct 19, 2022
Add _peek_bufsize to NetlinkSocketBase so recv can use it

Bug-Url: #1045
Bug-Url: #1044
Bug-Url: #751
Bug-Url: #813
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant