Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Followup to PR 9088 comments #9659

Merged
merged 38 commits into from
Nov 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
9c72e56
simplify io/functions.cpp data source/sink factories
cwharris Aug 14, 2021
884bde6
Merge branch 'branch-21.10' of github.com:rapidsai/cudf into io-funct…
cwharris Aug 16, 2021
88e2399
remove filepath-related logic from csv and json readers
cwharris Aug 17, 2021
62b9520
remove filepath logic from avro, parquet, orc readers
cwharris Aug 17, 2021
fb01294
move range size padding calculation out of json/csv reader and in to …
cwharris Aug 18, 2021
d422aeb
remove filepaths from json reader
cwharris Aug 18, 2021
e0cac1d
replace json reader impl buffer member with local variable
cwharris Aug 19, 2021
dc23634
replace josn reader sources member with local variable
cwharris Aug 20, 2021
166c4d3
delete useless json reader wrapper class
cwharris Aug 20, 2021
b3ce527
delete unused arrow namespace declaration
cwharris Aug 20, 2021
5a997b8
replace json reader_impl mr member with argument
cwharris Aug 20, 2021
89163fb
replace json reader_impl options with local variable
cwharris Aug 20, 2021
7ff3f1a
remove unused json_reader::allow_newlines_in_strings_ member
cwharris Aug 20, 2021
5c95398
replace json reader_impl::opts_ with local variable
cwharris Aug 20, 2021
0bde0b1
remove json reader_impl byte_range members in place of local variables
cwharris Aug 20, 2021
b4843f5
replace json reader_impl::dtypes_ with local variable
cwharris Aug 20, 2021
f511e68
replace json reader_impl::metadata_ with local variable
cwharris Aug 20, 2021
5e307b5
replace json reader_impl::data_ with local variable
cwharris Aug 20, 2021
f84dcc3
replace json::reader_impl column_map members with local variables
cwharris Aug 21, 2021
e11e9db
change json::reader_impl host buffer type from uint8_t to char
cwharris Aug 21, 2021
df10217
replace json::reader_impl uncomp data members with single span member
cwharris Aug 21, 2021
0417be8
json::reader_impl simplify device data copy logic
cwharris Aug 21, 2021
f0fd5c1
remove json::reader_impl::decompress_input function and inline the logic
cwharris Aug 21, 2021
45714bb
replace json::reader_impl::uncomp_data_ member with local variable
cwharris Aug 21, 2021
8d18ffe
relocate json::reader_impl decompression code
cwharris Aug 21, 2021
06a39b7
remove unneccessary json::reader_impl class
cwharris Aug 21, 2021
c658af1
remove deprected file header format
cwharris Aug 21, 2021
a1945af
remove json_common.h
cwharris Aug 21, 2021
e4b1621
Merge branch 'branch-21.10' of github.com:rapidsai/cudf into io-simpl…
cwharris Aug 24, 2021
cf5867f
re-delete json reader_impl.hpp
cwharris Aug 25, 2021
bd69fbd
fix bad merge where changes in 9079 were deleted.
cwharris Aug 25, 2021
33d3cb7
read_json: fix missing visitor_overload include
cwharris Aug 26, 2021
008265b
Merge branch 'branch-21.12' into io-simplify-json
cwharris Oct 26, 2021
45df102
Merge branch 'branch-21.12' of github.com:rapidsai/cudf into io-simpl…
cwharris Oct 27, 2021
542d797
Merge branch 'branch-21.12' of github.com:rapidsai/cudf into io-simpl…
cwharris Nov 8, 2021
5e0ff9a
remove unnecessary doc comments and div-by-1
cwharris Nov 11, 2021
2a97585
Merge branch 'branch-21.12' of github.com:rapidsai/cudf into io-simpl…
cwharris Nov 11, 2021
cf42838
fix formatting issue
cwharris Nov 11, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions cpp/include/cudf/io/detail/json.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,6 @@
* limitations under the License.
*/

/**
* @file json.hpp
* @brief cuDF-IO reader classes API
*/

#pragma once

#include <cudf/io/json.hpp>
Expand All @@ -33,10 +28,10 @@ namespace json {
/**
* @brief Reads and returns the entire data set.
*
* @param[in] sources Input `datasource` objects to read the dataset from
* @param[in] options Settings for controlling reading behavior
* @param[in] stream CUDA stream used for device memory operations and kernel launches
* @param[in] mr Device memory resource to use for device memory allocation
* @param sources Input `datasource` objects to read the dataset from
* @param options Settings for controlling reading behavior
* @param stream CUDA stream used for device memory operations and kernel launches
* @param mr Device memory resource to use for device memory allocation
*
* @return cudf::table object that contains the array of cudf::column.
*/
Expand Down
2 changes: 1 addition & 1 deletion cpp/src/io/json/reader_impl.cu
Original file line number Diff line number Diff line change
Expand Up @@ -345,7 +345,7 @@ std::pair<std::vector<std::string>, col_map_ptr_type> get_column_names_and_map(
rmm::cuda_stream_view stream)
{
// If file only contains one row, use the file size for the row size
uint64_t first_row_len = d_data.size() / sizeof(char);
uint64_t first_row_len = d_data.size();
bdice marked this conversation as resolved.
Show resolved Hide resolved
if (rec_starts.size() > 1) {
// Set first_row_len to the offset of the second row, if it exists
CUDA_TRY(cudaMemcpyAsync(&first_row_len,
Expand Down