prov/util: Fix setting of mr_mode #5667
Conversation
If the app passes FI_MR_VIRT_ADDR as hints, and provider supports FI_MR_BASIC, FI_MR_VIRT_ADDR will be removed in function fi_alter_domain_attr. Therefore, the created ofi_mr_map won't be able to support FI_MR_VIRT_ADDR. Fix such issue by substituting FI_MR_BASIC with FI_MR_VIRT_ADDR, FI_MR_ALLOCATED, and FI_MR_PROV_KEY. Signed-off-by: Jie Zhang <[email protected]>
|
Fix issue #5659 |
| @@ -1089,6 +1089,10 @@ static void fi_alter_domain_attr(struct fi_domain_attr *attr, | |||
| FI_MR_BASIC : FI_MR_SCALABLE; | |||
| } else { | |||
| if ((hints_mr_mode & attr->mr_mode) != attr->mr_mode) { | |||
| if (attr->mr_mode & FI_MR_BASIC) { | |||
| attr->mr_mode &= ~FI_MR_BASIC; | |||
| attr->mr_mode |= FI_MR_ALLOCATED | FI_MR_PROV_KEY | FI_MR_VIRT_ADDR; | |||
There was a problem hiding this comment.
The provider should indicate what mr_mode bits it requires. It may also indicate if it can handle BASIC and/or SCALABLE. If the app requests BASIC, then only BASIC may be returned, not the individual bits, which the app may not know how to interpret.
There was a problem hiding this comment.
Yeah, this is a requirement and the util code is doing the right thing by unsetting the bits that a provider doesn't require.
There was a problem hiding this comment.
@rajachan - I didn't completely follow your comment. Are you saying that the current code is correct?
There was a problem hiding this comment.
I was agreeing with you that providers should set the bits they require and the util code should not have to do what this PR is doing. I haven't looked at the shm/uber test failure that necessitated this change yet, but looks like that could be a provider/test bug.
Edit: #5672 makes more sense.
|
Close this as #5672 has been merged. |
If the app passes FI_MR_VIRT_ADDR as hints, and provider
supports FI_MR_BASIC, then FI_MR_VIRT_ADDR will be removed
when calling function fi_alter_domain_attr. Therefore, the created
ofi_mr_map won't be able to support FI_MR_VIRT_ADDR.
Fix such issue by substituting FI_MR_BASIC with FI_MR_VIRT_ADDR,
FI_MR_ALLOCATED, and FI_MR_PROV_KEY.
Signed-off-by: Jie Zhang [email protected]