smsc/XPMEM region map fails due to padding/alignment and/or inaccurate max address #10121
Comments
|
I am fighting an issue in smsc as well. I believe the +1 here is incorrect. Removing it doesn't change the outcome though. |
|
@gkatev Do your attachments succeed if you choose a smaller (4K) alignment via |
|
I'm havin a harder time reproducing it now, but given that in my debug code, where every time the 8MB-aligned attachment failed the 4KB one did succeed, I would guesstimate that it would not appear with a smaller alignment. Yes, it also seemed to me that that +1 is redundant. Perhaps also the -1? What issue are you seeing with smsc? Edit: Was able to reproduce it again with align=23, did not trigger align=12. |
|
OK, I think I am hitting the same issue. Will work on a patch. Am I right assuming that we shouldn't attempt to include the |
|
Yes it seems like a good solution for this case. (assuming that there's not reason to attach to another process's Maybe in order to cover all cases, one would have to take into account also the gaps in the remote process's memory map. Eg. in the example above, could the padding cause an attachment to exceed the heap's limit? But I guess that creating and keeping up to date such a structure is a more complicated endeavour without that much of a benefit. The neatest way to handle all cases would probably be to have the alignment be equal to the page size, seeing as it is the implicit padding that triggers the issue -- if the remote process communicated an address for another to attach to, it would/should be valid to attach to the (entire) page containing it. As I understand the padding is an optimization to proactively create an attachment to a larger area (?), as it might happen to contain a buffer that will be used in future communication. So, removing the padding might not be the optimal solution, as it might cause a performance loss in specific applications (but, I think, actually not because of the extra attach call, but rather in the case that a merge between two regions is triggered, and the existing one has to be detached and re-built). While we are at it, I would also suggest setting to minimum value for the alignment parameter not to 4K, but to the actual page size (eg. on an ARM system of mine it is 64K), as AFAIK xpmem will always align to the page. A middle-ground defensive solution to all this (along with not considering |
|
Yes as far as I can tell that solves it! I would probably have the bound fall back to the next page limit from the requested bound -- not sure if using the exact requested bound is safer than that or not. If the bound passed to xpmem_attach is not page-aligned, xpmem will align it and include the whole page in the attachment. The downside to that is that smsc/xpmem won't know about it, which could lead to duplicate attachments. But functionally, I'd say both options are okay. |
|
As far as I can tell this was fully fixed in the mentioned PRs, thanks! |
Hi, I observed that some of my SMSC
map_peer_regioncalls were failing. It appears that the 8 MB padding/alignment added to XPMEM attachments, in conjunction with an inaccurately(?) setaddress_max, brings the attachment range over the limit.Adding these debug prints:
Generates this output:
The attachment with size 8388609 ie. 0x800001 (btw, why the +1?) fails, whereas the 4096 one succeeds.
0x7fffff800000 + 0x800001 = 0x800000000001
If I'm interpreting the memory maps below correctly,
address_maxis set (incorrectly/inaccurately?) to0xffffffffff600fffaccording to[vsyscall]. The area for which themap_peer_regioncall is made is part of the stack.The stack here appears to be at
7ffdd10be000-7ffdd10e0000, but this may not actually be relevant as this the local process's map and not the remote one's? I can only imagine if I actually got the map of the remote process0x800000000001, would be outside the range, while0x7fffff800000 + 0x1000 = 0x7fffff801000would be within it./proc/self/mapscontent (of process callingxpmem_attach):The text was updated successfully, but these errors were encountered: