-
-
Notifications
You must be signed in to change notification settings - Fork 265
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement selection vector I/O with collective chunk filling #3826
Changes from 2 commits
35fadc4
f018960
5b501b9
be8f0e4
b919f73
6d10906
0f30b4f
9059377
abdc0c0
69f61b5
d7786bf
3590161
22a2bcb
22fdd88
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2473,8 +2473,13 @@ H5FD__mpio_write_vector(H5FD_t *_file, hid_t H5_ATTR_UNUSED dxpl_id, uint32_t co | |
HGOTO_ERROR(H5E_VFL, H5E_CANTGET, FAIL, "can't build MPI datatypes for I/O"); | ||
|
||
/* Compute max address written to */ | ||
if (count > 0) | ||
max_addr = s_addrs[count - 1] + (haddr_t)(s_sizes[count - 1]); | ||
/* Bug: need to account for fixed size optimization */ | ||
if (count > 0) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A few things here:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also the comment shouldn't proclaim it as a bug, just explain what it's doing There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually my code is wrong too, I'll try to fix it later tonight. We might need to change the H5FD__mpio_vector_build_types function There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think <s_sizes> returned from H5FD__mpio_vector_build_types() can be (1) or (2) as follows:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It can be any number of entries between 1 (with a 0 terminator if count > 1) and count There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we have time it would be great to add a test for this case. We would have to use the public H5FD interface There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Applied the patch from Neil. |
||
uint32_t idx = count - 1; | ||
if (s_sizes[0] != 0 && s_sizes[1] == 0) | ||
idx = 0; | ||
max_addr = s_addrs[count - 1] + (haddr_t)(s_sizes[idx]); | ||
} | ||
|
||
/* free sorted vectors if they exist */ | ||
if (!vector_was_sorted) { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -556,10 +556,20 @@ verify_chunk_opt_status(size_t num_dsets, test_mode_t test_mode, bool any_io, bo | |
* elements (because no elements were raw data), which is what happens when performing I/O on a | ||
* filtered dataset with no selection. Vector I/O does report an I/O call was made if passed a raw | ||
* data element of size 0, so this is consistent. */ | ||
/* Changes: for allocation, the reported I/O is vector I/O */ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No need to call out "changes" in the code comment here. Just explain why we expect the different modes under each condition as the code stands with this PR |
||
if (!any_io) { | ||
if (did_alloc || (num_dsets > 0 && test_mode == USE_MULTIPLE_DATASETS_MIXED_FILTERED)) | ||
if (did_alloc && (num_dsets > 0 && test_mode == USE_MULTIPLE_DATASETS_MIXED_FILTERED)) { | ||
VRFY((H5D_VECTOR_IO | H5D_SCALAR_IO) == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was vector and scalar I/O"); | ||
} | ||
else if (did_alloc) { | ||
VRFY(H5D_VECTOR_IO == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was vector I/O"); | ||
} | ||
else if (num_dsets > 0 && test_mode == USE_MULTIPLE_DATASETS_MIXED_FILTERED) { | ||
VRFY(H5D_SCALAR_IO == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was scalar I/O"); | ||
} | ||
else | ||
VRFY(0 == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was 0 (no I/O)"); | ||
|
@@ -592,15 +602,14 @@ verify_chunk_opt_status(size_t num_dsets, test_mode_t test_mode, bool any_io, bo | |
* should be scalar I/O for allocation in addition to vector I/O for the actual data. | ||
* If we're reading from an unallocated dataset then there should be no actual I/O. | ||
* Otherwise there should only be vector I/O. */ | ||
if (did_alloc) | ||
VRFY((H5D_SCALAR_IO | H5D_VECTOR_IO) == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was scalar and vector I/O"); | ||
else if (unalloc_read) | ||
/* Changes: for allocation, the reported I/O is vector I/O */ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here. Explain that both collective filling/allocation and filtered I/O use vector I/O so only that is reported whether or not allocation happened. |
||
if (unalloc_read) | ||
VRFY(0 == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was 0 (no I/O)"); | ||
else | ||
else { /* did_alloc || !unalloc_read */ | ||
VRFY(H5D_VECTOR_IO == actual_sel_io_mode_reduced, | ||
"verified actual selection I/O mode was vector I/O"); | ||
} | ||
break; | ||
|
||
case USE_MULTIPLE_DATASETS_MIXED_FILTERED: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comment: I'd prefer to just always call H5MM_xfree here to make the code more robust against future changes. It's a no-op if io_sizes in NULL so this won't cause problems
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The C standard explicitly says that calling free() on NULL pointers is acceptable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed