Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve parameters of cuIO kernels (Avro) #5710

Closed
vuule opened this issue Jul 16, 2020 · 4 comments
Closed

Improve parameters of cuIO kernels (Avro) #5710

vuule opened this issue Jul 16, 2020 · 4 comments
Labels
cuIO cuIO issue

Comments

@vuule
Copy link
Contributor

vuule commented Jul 16, 2020

Many cuIO kernels and their callers have a large number of parameters. These functions can the improved by grouping their parameters into meaningful abstractions.

Example:

uint32_t gather_row_offsets(uint64_t *row_ctx,
uint64_t *offsets_out,
const char *start,
size_t chunk_size,
size_t parse_pos,
size_t start_offset,
size_t data_size,
size_t byte_range_start,
size_t byte_range_end,
size_t skip_rows,
size_t num_row_offsets,
const cudf::io::ParseOptions &options,
cudaStream_t stream = 0);

Edit: parameter order is also inconsistent/incorrect - in some places out parameters are places before inputs.

@vuule
Copy link
Contributor Author

vuule commented Aug 12, 2020

The changes should include a type that is a non-owning view of a device buffer - something like device_span.
There are many places where a pointer to a buffer and the buffer element count are passed as separate parameters.

@cwharris
Copy link
Contributor

cwharris commented Sep 16, 2020

Introducing device_span in #6233 which should prove useful here.

@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@vuule
Copy link
Contributor Author

vuule commented Mar 22, 2021

Keeping the issue open for the existing Avro PR.
For the other components - we are improving the parameters and adding useful abstractions (like span, 2dspan, column views) as we make other refactorings in the code. It does not seem like we should make changes focused on the parameters now that the refactoring is well on it's way.

@vuule vuule changed the title Improve parameters of cuIO kernels Improve parameters of cuIO kernels (Avro) Mar 30, 2021
@vuule vuule closed this as completed Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuIO cuIO issue
Projects
None yet
Development

No branches or pull requests

2 participants