Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert cudf::concatenate APIs to use spans and device_uvector #7621

Merged
merged 22 commits into from
Mar 25, 2021

Conversation

harrism
Copy link
Member

@harrism harrism commented Mar 17, 2021

Contributes to #7287

This PR replaces std::vector with host_span in public and detail cudf::contatenate functions, and replaces rmm::device_vector with rmm::device_uvector in the concatenate implementations.

It also strengthens the SFINAE restrictions on cudf::host_span and cudf::device_span so that they cannot be constructed from containers unless the container's value_type is the same as the span's value_type.

This PR also

  • Updates cython.
  • benchmarks before and after

@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Mar 17, 2021
@harrism harrism added breaking Breaking change improvement Improvement / enhancement to an existing function labels Mar 17, 2021
@harrism harrism self-assigned this Mar 17, 2021
@github-actions github-actions bot added the Python Affects Python cuDF API. label Mar 23, 2021
@harrism harrism added 3 - Ready for Review Ready for review by team 4 - Needs Review Waiting for reviewer to review or respond labels Mar 23, 2021
@harrism harrism marked this pull request as ready for review March 23, 2021 05:24
@harrism harrism requested review from a team as code owners March 23, 2021 05:24
@codecov
Copy link

codecov bot commented Mar 23, 2021

Codecov Report

Merging #7621 (0a330f1) into branch-0.19 (7871e7a) will increase coverage by 0.70%.
The diff coverage is n/a.

Impacted file tree graph

@@               Coverage Diff               @@
##           branch-0.19    #7621      +/-   ##
===============================================
+ Coverage        81.86%   82.57%   +0.70%     
===============================================
  Files              101      101              
  Lines            16884    17551     +667     
===============================================
+ Hits             13822    14492     +670     
+ Misses            3062     3059       -3     
Impacted Files Coverage Δ
python/cudf/cudf/utils/gpu_utils.py 53.65% <0.00%> (-4.88%) ⬇️
python/cudf/cudf/core/column/decimal.py 92.75% <0.00%> (-2.12%) ⬇️
python/cudf/cudf/core/column/lists.py 90.00% <0.00%> (-1.40%) ⬇️
python/cudf/cudf/core/column/numerical.py 94.83% <0.00%> (-0.20%) ⬇️
python/cudf/cudf/core/column/column.py 87.61% <0.00%> (-0.15%) ⬇️
python/cudf/cudf/utils/utils.py 85.36% <0.00%> (-0.07%) ⬇️
python/cudf/cudf/io/feather.py 100.00% <0.00%> (ø)
python/cudf/cudf/comm/serialize.py 0.00% <0.00%> (ø)
python/cudf/cudf/_fuzz_testing/io.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/column/struct.py 100.00% <0.00%> (ø)
... and 43 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de55832...0a330f1. Read the comment docs.

Copy link
Contributor

@vuule vuule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few very minor suggestions.

cpp/src/copying/concatenate.cu Outdated Show resolved Hide resolved
cpp/src/copying/concatenate.cu Outdated Show resolved Hide resolved
cpp/src/dictionary/add_keys.cu Outdated Show resolved Hide resolved
cpp/src/dictionary/set_keys.cu Outdated Show resolved Hide resolved
cpp/src/interop/from_arrow.cpp Outdated Show resolved Hide resolved
@harrism
Copy link
Member Author

harrism commented Mar 23, 2021

This change helps more than I expected. Concatenate is significantly faster now, except for concatenating strings columns, where the change seems to be noise.

 (rapids) rapids@compose:~/cudf/cpp/build$ ./release/_deps/benchmark-src/tools/compare.py benchmarks concat_vector.json concat_uvector.json 
Comparing concat_vector.json to concat_uvector.json
Benchmark                                                                                            Time             CPU      Time Old      Time New       CPU Old       CPU New
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/2/manual_time                        -0.1255         -0.0876             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/2/manual_time                       -0.1295         -0.0969             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/2/manual_time                      -0.1726         -0.1245             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/2/manual_time                     -0.1697         -0.1278             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/2/manual_time                    -0.1630         -0.1417             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/8/manual_time                        -0.7812         -0.5405             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/8/manual_time                       -0.8570         -0.6873             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/8/manual_time                      -0.7063         -0.4567             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/8/manual_time                     -0.7971         -0.6858             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/8/manual_time                    -0.2961         -0.2407             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/64/manual_time                       -0.7332         -0.4947             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/64/manual_time                      -0.7125         -0.4822             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/64/manual_time                     -0.5927         -0.4212             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/64/manual_time                    -0.2869         -0.2354             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/64/manual_time                   -0.0508         +0.0857             0             0             1             1
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/512/manual_time                      -0.4641         -0.3807             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/512/manual_time                     -0.4159         -0.3501             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/512/manual_time                    -0.3344         -0.0987             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/512/manual_time                   -0.0909         -0.0228             1             0             1             1
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/512/manual_time                  -0.1672         -0.1815             5             4             5             4
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/1024/manual_time                     -0.2869         -0.2447             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/1024/manual_time                    -0.2659         -0.2340             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/1024/manual_time                   -0.1258         -0.1139             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/1024/manual_time                  -0.0253         -0.0244             1             1             1             1
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/1024/manual_time                 -0.0202         -0.0215             8             7             8             7
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/2/manual_time                         -0.5505         -0.3431             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/2/manual_time                        -0.8175         -0.7028             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/2/manual_time                       -0.5401         -0.3530             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/2/manual_time                      -0.7764         -0.5351             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/2/manual_time                     -0.4697         -0.3557             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/8/manual_time                         -0.5538         -0.3450             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/8/manual_time                        -0.5501         -0.3640             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/8/manual_time                       -0.6089         -0.4234             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/8/manual_time                      -0.6215         -0.5252             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/8/manual_time                     -0.2549         +0.1315             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/64/manual_time                        -0.4908         -0.2063             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/64/manual_time                       -0.5375         -0.3868             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/64/manual_time                      -0.4666         -0.3379             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/64/manual_time                     -0.2138         -0.1729             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/64/manual_time                    -0.0465         -0.0418             1             0             1             1
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/512/manual_time                       -0.5753         -0.5316             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/512/manual_time                      -0.2500         -0.2037             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/512/manual_time                     -0.1840         -0.1569             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/512/manual_time                    -0.0680         -0.0681             1             1             1             1
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/512/manual_time                   -0.0600         -0.0651             4             4             4             4
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/1024/manual_time                      -0.2472         -0.2092             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/1024/manual_time                     -0.1582         -0.1147             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/1024/manual_time                    -0.0916         -0.0548             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/1024/manual_time                   -0.0239         -0.0205             1             1             1             1
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/1024/manual_time                  -0.0154         -0.0152             8             8             8             8
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/2/manual_time                      +0.0040         -0.0004             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/2/manual_time                      +0.0163         +0.0092             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/2/manual_time                     +0.0096         +0.0059             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/2/manual_time                      +0.0044         +0.0026             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/2/manual_time                      -0.0041         -0.0050             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/2/manual_time                     -0.0060         -0.0042             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/2/manual_time                     +0.0113         +0.0108             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/2/manual_time                     +0.0134         +0.0130             0             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/2/manual_time                    +0.0073         +0.0076             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/8/manual_time                      -0.7622         -0.5951             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/8/manual_time                      -0.7606         -0.5886             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/8/manual_time                     -0.7356         -0.5808             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/8/manual_time                      -0.6889         -0.6373             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/8/manual_time                      -0.7042         -0.6167             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/8/manual_time                     -0.7063         -0.6648             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/8/manual_time                     -0.7126         -0.6465             1             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/8/manual_time                     -0.6820         -0.6530             1             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/8/manual_time                    -0.6907         -0.6705             1             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/64/manual_time                     -0.6395         -0.5066             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/64/manual_time                     -0.6425         -0.5170             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/64/manual_time                    -0.6162         -0.5149             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/64/manual_time                     -0.6008         -0.5382             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/64/manual_time                     -0.5977         -0.5670             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/64/manual_time                    -0.6446         -0.6123             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/64/manual_time                    -0.5745         -0.5451             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/64/manual_time                    -0.5861         -0.5769             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/64/manual_time                   -0.6474         -0.6357             2             1             2             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/128/manual_time                    -0.5515         -0.4275             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/128/manual_time                    -0.5578         -0.4656             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/128/manual_time                   -0.5367         -0.4536             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/128/manual_time                    -0.5226         -0.4986             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/128/manual_time                    -0.5357         -0.5123             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/128/manual_time                   -0.5783         -0.5551             0             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/128/manual_time                   -0.5238         -0.5166             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/128/manual_time                   -0.5346         -0.5276             2             1             2             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/128/manual_time                  -0.6031         -0.5970             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/2/manual_time                       -0.5976         -0.4964             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/2/manual_time                       -0.5444         -0.4370             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/2/manual_time                      -0.5910         -0.4909             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/2/manual_time                       -0.5986         -0.5683             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/2/manual_time                       -0.5419         -0.5093             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/2/manual_time                      -0.5102         -0.4822             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/2/manual_time                      -0.6232         -0.6154             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/2/manual_time                      -0.5656         -0.5578             1             1             1             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/2/manual_time                     -0.5317         -0.5252             1             1             1             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/8/manual_time                       -0.7337         -0.6613             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/8/manual_time                       -0.6282         -0.5379             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/8/manual_time                      -0.8005         -0.7547             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/8/manual_time                       -0.5191         -0.4921             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/8/manual_time                       -0.4846         -0.4590             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/8/manual_time                      -0.4801         -0.4597             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/8/manual_time                      -0.4604         -0.4543             1             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/8/manual_time                      -0.4603         -0.4544             1             1             1             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/8/manual_time                     -0.4731         -0.4673             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/64/manual_time                      -0.4620         -0.3874             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/64/manual_time                      -0.4396         -0.3713             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/64/manual_time                     -0.4133         -0.3494             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/64/manual_time                      -0.4165         -0.3997             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/64/manual_time                      -0.6208         -0.6062             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/64/manual_time                     -0.7292         -0.7183             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/64/manual_time                     -0.4574         -0.4539             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/64/manual_time                     -0.4674         -0.4630             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/64/manual_time                    -0.4047         -0.3997             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/128/manual_time                     -0.4175         -0.3564             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/128/manual_time                     -0.3917         -0.3388             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/128/manual_time                    -0.3557         -0.3113             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/128/manual_time                     -0.3754         -0.3615             0             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/128/manual_time                     -0.3863         -0.3734             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/128/manual_time                    -0.3653         -0.3534             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/128/manual_time                    -0.4628         -0.4600             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/128/manual_time                    -0.3721         -0.3680             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/128/manual_time                   -0.3267         -0.3236             2             2             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/2/manual_time                      -0.0664         -0.0652             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/2/manual_time                      -0.2207         -0.2154             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/2/manual_time                     +0.0032         +0.0031             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/2/manual_time                    +0.0036         +0.0083             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/2/manual_time                     -0.0088         -0.0082             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/2/manual_time                     -0.0135         -0.0112             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/2/manual_time                    +0.0063         +0.0066             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/2/manual_time                   -0.0019         -0.0026             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/2/manual_time                    -0.0070         -0.0053             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/2/manual_time                    -0.0076         -0.0075             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/2/manual_time                   -0.0015         -0.0014             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/2/manual_time                  +0.0018         +0.0041             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/8/manual_time                      -0.0031         -0.0029             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/8/manual_time                      -0.0169         -0.0145             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/8/manual_time                     -0.0203         -0.0162             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/8/manual_time                    -0.0103         -0.0108             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/8/manual_time                     -0.0086         -0.0066             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/8/manual_time                     -0.0048         -0.0035             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/8/manual_time                    -0.0323         -0.0257             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/8/manual_time                   +0.0069         +0.0068             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/8/manual_time                    -0.0100         -0.0084             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/8/manual_time                    +0.0032         +0.0030             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/8/manual_time                   -0.0190         -0.0176             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/8/manual_time                  -0.0087         -0.0081             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/64/manual_time                     -0.0500         -0.0424             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/64/manual_time                     -0.0871         -0.0733             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/64/manual_time                    -0.0102         -0.0090             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/64/manual_time                   -0.0178         -0.0168             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/64/manual_time                    -0.0466         -0.0403             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/64/manual_time                    -0.0526         -0.0482             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/64/manual_time                   -0.0109         -0.0107             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/64/manual_time                  -0.0003         -0.0005             2             2             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/64/manual_time                   -0.0373         -0.0348             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/64/manual_time                   -0.0523         -0.0501             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/64/manual_time                  +0.0070         +0.0068             1             1             1             1
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/64/manual_time                 -0.0028         -0.0026             2             2             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/256/manual_time                    -0.0423         -0.0374             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/256/manual_time                    -0.0444         -0.0409             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/256/manual_time                   -0.0212         -0.0216             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/256/manual_time                  -0.0071         -0.0071             1             1             1             1
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/256/manual_time                   -0.0349         -0.0332             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/256/manual_time                   -0.0356         -0.0334             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/256/manual_time                  +0.0047         +0.0364             1             1             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/256/manual_time                 +0.0015         +0.0016             7             7             7             7
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/256/manual_time                  -0.0183         -0.0131             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/256/manual_time                  +0.0535         +0.0706             0             1             1             1
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/256/manual_time                 -0.0033         -0.0033             5             5             5             5
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/256/manual_time                -0.0069         -0.0070             8             8             8             8
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/2/manual_time                       -0.0082         -0.0080             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/2/manual_time                       -0.0089         -0.0093             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/2/manual_time                      -0.0022         -0.0020             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/2/manual_time                     -0.0103         -0.0090             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/2/manual_time                      -0.0023         -0.0025             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/2/manual_time                      -0.0107         -0.0106             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/2/manual_time                     -0.0231         -0.0206             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/2/manual_time                    -0.0350         -0.0322             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/2/manual_time                     -0.0265         -0.0244             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/2/manual_time                     -0.0135         -0.0121             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/2/manual_time                    -0.0157         -0.0130             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/2/manual_time                   -0.0377         -0.0344             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/8/manual_time                       -0.0122         -0.0114             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/8/manual_time                       -0.0115         -0.0120             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/8/manual_time                      -0.0170         -0.0160             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/8/manual_time                     -0.0083         -0.0093             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/8/manual_time                      -0.0029         -0.0025             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/8/manual_time                      +0.0201         +0.0174             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/8/manual_time                     -0.0097         -0.0084             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/8/manual_time                    -0.0063         -0.0090             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/8/manual_time                     -0.0088         -0.0064             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/8/manual_time                     -0.0124         -0.0130             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/8/manual_time                    -0.0231         -0.0237             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/8/manual_time                   -0.0167         -0.0164             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/64/manual_time                      -0.0212         -0.0182             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/64/manual_time                      -0.0200         -0.0177             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/64/manual_time                     -0.0227         -0.0211             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/64/manual_time                    -0.0181         -0.0170             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/64/manual_time                     -0.0242         -0.0210             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/64/manual_time                     -0.0398         -0.0389             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/64/manual_time                    -0.0076         -0.0096             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/64/manual_time                   -0.0055         -0.0070             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/64/manual_time                    -0.0175         -0.0159             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/64/manual_time                    -0.0185         -0.0195             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/64/manual_time                   -0.0077         -0.0091             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/64/manual_time                  -0.0063         -0.0074             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/256/manual_time                     -0.0509         -0.0459             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/256/manual_time                     -0.0509         -0.0475             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/256/manual_time                    -0.0356         -0.0334             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/256/manual_time                   -0.0157         -0.0169             1             0             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/256/manual_time                    -0.0568         -0.0534             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/256/manual_time                    -0.0748         -0.0809             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/256/manual_time                   -0.0291         -0.0396             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/256/manual_time                  -0.0059         -0.0107             2             2             2             2
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/256/manual_time                   -0.0384         -0.0347             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/256/manual_time                   -0.0325         -0.0285             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/256/manual_time                  -0.0082         -0.0078             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/256/manual_time                 +0.0538         +0.0722             4             4             4             4

@vuule
Copy link
Contributor

vuule commented Mar 23, 2021

@harrism Does the perf chart show relative difference?

@harrism
Copy link
Member Author

harrism commented Mar 24, 2021

@vuule yes they are relative. The computation is (new_time - old_time) / |old_time|.

With my latest commit I have also replaced device_vector with device_uvector in strings concatenate, which I missed before. Also fused a transform and inclusive_scan that according to a comment would not previously compile. Let's see if that passes CI. Strings concatenate is now up to >60% faster (small columns), and is faster across the board.

(rapids) rapids@compose:~/cudf/cpp/build$ ./release/_deps/benchmark-src/tools/compare.py benchmarks concat_vector.json concat_string_uvector2.json 
Comparing concat_vector.json to concat_string_uvector2.json
Benchmark                                                                                            Time             CPU      Time Old      Time New       CPU Old       CPU New
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/2/manual_time                        +0.1495         +0.1056             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/2/manual_time                       -0.0843         -0.0620             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/2/manual_time                      -0.1394         -0.1031             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/2/manual_time                     -0.0106         -0.0105             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/2/manual_time                    -0.1456         -0.1323             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/8/manual_time                        -0.7813         -0.5293             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/8/manual_time                       -0.8607         -0.6854             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/8/manual_time                      -0.7077         -0.4587             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/8/manual_time                     -0.7952         -0.6720             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/8/manual_time                    -0.2977         -0.2403             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/64/manual_time                       -0.7386         -0.4936             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/64/manual_time                      -0.7099         -0.4755             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/64/manual_time                     -0.5888         -0.4149             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/64/manual_time                    -0.2873         -0.2296             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/64/manual_time                   -0.0517         -0.0491             0             0             1             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/512/manual_time                      -0.4509         -0.3752             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/512/manual_time                     -0.4312         -0.3728             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/512/manual_time                    -0.2978         -0.2765             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/512/manual_time                   -0.0806         -0.0837             1             1             1             1
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/512/manual_time                  -0.1828         -0.1997             5             4             5             4
Concatenate<int64_t, false>/concat_columns_int64_non_null/64/1024/manual_time                     -0.2711         -0.2312             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/512/1024/manual_time                    -0.2625         -0.2357             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/4096/1024/manual_time                   -0.1268         -0.1156             0             0             0             0
Concatenate<int64_t, false>/concat_columns_int64_non_null/32768/1024/manual_time                  -0.0272         -0.0076             1             1             1             1
Concatenate<int64_t, false>/concat_columns_int64_non_null/262144/1024/manual_time                 -0.0207         -0.0220             8             7             8             7
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/2/manual_time                         -0.5441         -0.3442             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/2/manual_time                        -0.8177         -0.7044             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/2/manual_time                       -0.5513         -0.3637             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/2/manual_time                      -0.7769         -0.6605             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/2/manual_time                     -0.4564         -0.3265             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/8/manual_time                         -0.5407         -0.3547             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/8/manual_time                        -0.5450         -0.3455             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/8/manual_time                       -0.6014         -0.4229             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/8/manual_time                      -0.6204         -0.5294             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/8/manual_time                     -0.2542         -0.1970             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/64/manual_time                        -0.4943         -0.3300             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/64/manual_time                       -0.5269         -0.3765             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/64/manual_time                      -0.4646         -0.3386             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/64/manual_time                     -0.2085         -0.1664             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/64/manual_time                    -0.0495         -0.0402             1             0             1             1
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/512/manual_time                       -0.5613         -0.4887             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/512/manual_time                      -0.2489         -0.2037             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/512/manual_time                     -0.1644         -0.1178             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/512/manual_time                    -0.0648         -0.0612             1             1             1             1
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/512/manual_time                   -0.0465         -0.0450             4             4             4             4
Concatenate<int64_t, true>/concat_columns_int64_nullable/64/1024/manual_time                      -0.1994         -0.1616             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/512/1024/manual_time                     -0.1548         -0.1319             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/4096/1024/manual_time                    -0.1156         -0.0030             0             0             0             0
Concatenate<int64_t, true>/concat_columns_int64_nullable/32768/1024/manual_time                   -0.0237         -0.0208             1             1             1             1
Concatenate<int64_t, true>/concat_columns_int64_nullable/262144/1024/manual_time                  -0.0053         -0.0047             8             8             8             8
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/2/manual_time                      +0.0258         +0.0198             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/2/manual_time                      +0.0006         +0.0013             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/2/manual_time                     +0.0453         +0.0340             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/2/manual_time                      +0.0209         +0.0185             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/2/manual_time                      +0.0180         +0.0162             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/2/manual_time                     +0.0396         +0.0384             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/2/manual_time                     +0.0548         +0.0531             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/2/manual_time                     +0.0447         +0.0441             0             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/2/manual_time                    +0.0766         +0.0748             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/8/manual_time                      -0.7656         -0.5855             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/8/manual_time                      -0.7631         -0.5900             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/8/manual_time                     -0.7300         -0.5758             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/8/manual_time                      -0.6862         -0.6310             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/8/manual_time                      -0.6959         -0.6490             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/8/manual_time                     -0.6946         -0.6529             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/8/manual_time                     -0.6809         -0.6688             1             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/8/manual_time                     -0.6701         -0.6528             1             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/8/manual_time                    -0.6864         -0.6743             1             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/64/manual_time                     -0.6417         -0.5125             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/64/manual_time                     -0.6327         -0.5086             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/64/manual_time                    -0.6044         -0.5111             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/64/manual_time                     -0.6032         -0.5704             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/64/manual_time                     -0.5841         -0.5538             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/64/manual_time                    -0.6465         -0.6128             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/64/manual_time                    -0.5796         -0.5725             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/64/manual_time                    -0.5821         -0.5728             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/64/manual_time                   -0.6402         -0.6314             2             1             2             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/2/128/manual_time                    -0.5441         -0.4478             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/2/128/manual_time                    -0.5435         -0.4523             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/2/128/manual_time                   -0.5458         -0.4598             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/8/128/manual_time                    -0.4898         -0.4673             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/8/128/manual_time                    -0.5075         -0.4855             0             0             0             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/8/128/manual_time                   -0.5768         -0.5533             0             0             1             0
Concatenate<int64_t, false>/concat_tables_int64_non_null/256/32/128/manual_time                   -0.4863         -0.4793             1             1             1             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/512/32/128/manual_time                   -0.5150         -0.5081             2             1             2             1
Concatenate<int64_t, false>/concat_tables_int64_non_null/4096/32/128/manual_time                  -0.5980         -0.5922             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/2/manual_time                       -0.5915         -0.4898             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/2/manual_time                       -0.5345         -0.4284             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/2/manual_time                      -0.5843         -0.4869             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/2/manual_time                       -0.5940         -0.5638             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/2/manual_time                       -0.5323         -0.4999             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/2/manual_time                      -0.4865         -0.4605             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/2/manual_time                      -0.6173         -0.6098             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/2/manual_time                      -0.5475         -0.5399             1             1             1             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/2/manual_time                     -0.5127         -0.5065             1             1             1             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/8/manual_time                       -0.7231         -0.6520             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/8/manual_time                       -0.6048         -0.5182             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/8/manual_time                      -0.7934         -0.7484             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/8/manual_time                       -0.5255         -0.4986             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/8/manual_time                       -0.4780         -0.4529             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/8/manual_time                      -0.4787         -0.4590             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/8/manual_time                      -0.4536         -0.4475             1             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/8/manual_time                      -0.4565         -0.4503             1             1             1             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/8/manual_time                     -0.4668         -0.4608             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/64/manual_time                      -0.4551         -0.3827             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/64/manual_time                      -0.4472         -0.3781             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/64/manual_time                     -0.4295         -0.3719             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/64/manual_time                      -0.4108         -0.3929             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/64/manual_time                      -0.6366         -0.6247             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/64/manual_time                     -0.7252         -0.7140             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/64/manual_time                     -0.4367         -0.4332             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/64/manual_time                     -0.4526         -0.4483             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/64/manual_time                    -0.4083         -0.4033             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/2/128/manual_time                     -0.3932         -0.3355             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/2/128/manual_time                     -0.3836         -0.3305             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/2/128/manual_time                    -0.3306         -0.2848             0             0             0             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/8/128/manual_time                     -0.3634         -0.3505             0             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/8/128/manual_time                     -0.3603         -0.3481             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/8/128/manual_time                    -0.3525         -0.3408             1             0             1             0
Concatenate<int64_t, true>/concat_tables_int64_nullable/256/32/128/manual_time                    -0.4414         -0.4383             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/512/32/128/manual_time                    -0.3563         -0.3526             2             1             2             1
Concatenate<int64_t, true>/concat_tables_int64_nullable/4096/32/128/manual_time                   -0.2921         -0.2891             2             2             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/2/manual_time                      -0.5892         -0.5056             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/2/manual_time                      -0.6616         -0.5896             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/2/manual_time                     -0.5587         -0.4713             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/2/manual_time                    -0.5544         -0.4684             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/2/manual_time                     -0.5594         -0.4709             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/2/manual_time                     -0.5605         -0.4724             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/2/manual_time                    -0.5310         -0.4528             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/2/manual_time                   -0.3126         -0.2764             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/2/manual_time                    -0.5532         -0.4643             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/2/manual_time                    -0.5569         -0.4718             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/2/manual_time                   -0.3114         -0.2762             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/2/manual_time                  -0.3135         -0.2803             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/8/manual_time                      -0.5610         -0.4730             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/8/manual_time                      -0.5613         -0.4735             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/8/manual_time                     -0.5519         -0.4708             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/8/manual_time                    -0.5248         -0.4607             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/8/manual_time                     -0.5509         -0.4664             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/8/manual_time                     -0.5414         -0.4616             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/8/manual_time                    -0.4706         -0.4197             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/8/manual_time                   -0.1367         -0.1288             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/8/manual_time                    -0.5385         -0.4585             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/8/manual_time                    -0.5322         -0.4564             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/8/manual_time                   -0.1382         -0.1282             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/8/manual_time                  -0.1408         -0.1325             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/64/manual_time                     -0.5091         -0.4319             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/64/manual_time                     -0.5354         -0.4622             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/64/manual_time                    -0.4122         -0.3716             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/64/manual_time                   -0.2351         -0.2206             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/64/manual_time                    -0.4702         -0.4159             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/64/manual_time                    -0.4302         -0.3883             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/64/manual_time                   -0.1682         -0.1611             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/64/manual_time                  -0.0260         -0.0258             2             2             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/64/manual_time                   -0.4127         -0.3736             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/64/manual_time                   -0.3609         -0.3322             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/64/manual_time                  -0.0208         -0.0206             1             1             1             1
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/64/manual_time                 -0.0174         -0.0173             2             2             2             2
ConcatenateStrings<false>/concat_string_columns_non_null/256/8/256/manual_time                    -0.3834         -0.3475             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/8/256/manual_time                    -0.3588         -0.3307             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/8/256/manual_time                   -0.1943         -0.1863             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/16384/8/256/manual_time                  -0.0724         -0.0726             1             1             1             1
ConcatenateStrings<false>/concat_string_columns_non_null/256/64/256/manual_time                   -0.2794         -0.2615             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/64/256/manual_time                   -0.2136         -0.2041             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/64/256/manual_time                  -0.0437         -0.0438             1             1             2             1
ConcatenateStrings<false>/concat_string_columns_non_null/16384/64/256/manual_time                 -0.0089         -0.0090             7             7             7             7
ConcatenateStrings<false>/concat_string_columns_non_null/256/128/256/manual_time                  -0.2106         -0.1991             0             0             0             0
ConcatenateStrings<false>/concat_string_columns_non_null/512/128/256/manual_time                  -0.1384         -0.1344             0             0             1             0
ConcatenateStrings<false>/concat_string_columns_non_null/4096/128/256/manual_time                 +0.0149         +0.0149             5             5             5             5
ConcatenateStrings<false>/concat_string_columns_non_null/16384/128/256/manual_time                -0.0078         -0.0079             8             8             8             8
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/2/manual_time                       -0.3915         -0.3150             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/2/manual_time                       -0.2925         -0.2029             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/2/manual_time                      -0.4691         -0.4011             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/2/manual_time                     -0.4888         -0.4229             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/2/manual_time                      -0.5039         -0.4307             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/2/manual_time                      -0.5122         -0.4385             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/2/manual_time                     -0.4984         -0.4333             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/2/manual_time                    -0.4686         -0.4109             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/2/manual_time                     -0.5206         -0.4466             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/2/manual_time                     -0.4807         -0.4112             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/2/manual_time                    -0.5011         -0.4316             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/2/manual_time                   -0.4428         -0.3929             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/8/manual_time                       -0.5043         -0.4322             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/8/manual_time                       -0.4938         -0.4236             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/8/manual_time                      -0.4820         -0.4212             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/8/manual_time                     -0.4599         -0.4041             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/8/manual_time                      -0.4714         -0.4039             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/8/manual_time                      -0.4754         -0.4082             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/8/manual_time                     -0.4529         -0.3973             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/8/manual_time                    -0.3830         -0.3483             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/8/manual_time                     -0.4779         -0.4109             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/8/manual_time                     -0.4924         -0.4277             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/8/manual_time                    -0.4333         -0.3864             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/8/manual_time                   -0.3231         -0.2984             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/64/manual_time                      -0.4573         -0.3975             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/64/manual_time                      -0.4591         -0.4020             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/64/manual_time                     -0.4139         -0.3685             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/64/manual_time                    -0.3007         -0.2772             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/64/manual_time                     -0.4406         -0.3854             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/64/manual_time                     -0.4306         -0.3844             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/64/manual_time                    -0.2603         -0.2459             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/64/manual_time                   -0.1163         -0.1154             1             0             1             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/64/manual_time                    -0.4244         -0.3744             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/64/manual_time                    -0.3881         -0.3500             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/64/manual_time                   -0.1920         -0.1806             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/64/manual_time                  -0.0702         -0.0685             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/256/8/256/manual_time                     -0.3552         -0.3187             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/8/256/manual_time                     -0.3285         -0.2966             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/8/256/manual_time                    -0.2500         -0.2343             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/16384/8/256/manual_time                   -0.1213         -0.1180             1             0             1             0
ConcatenateStrings<true>/concat_string_columns_nullable/256/64/256/manual_time                    -0.3159         -0.2911             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/64/256/manual_time                    -0.2955         -0.2835             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/64/256/manual_time                   -0.1054         -0.1124             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/16384/64/256/manual_time                  -0.0302         -0.0326             2             2             2             2
ConcatenateStrings<true>/concat_string_columns_nullable/256/128/256/manual_time                   -0.2613         -0.2424             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/512/128/256/manual_time                   -0.2088         -0.1976             0             0             0             0
ConcatenateStrings<true>/concat_string_columns_nullable/4096/128/256/manual_time                  -0.0548         -0.0517             1             1             1             1
ConcatenateStrings<true>/concat_string_columns_nullable/16384/128/256/manual_time                 -0.0140         -0.0136             4             4             4             4

@harrism harrism requested a review from shwina March 24, 2021 10:15
Comment on lines 23 to 33
# these don't exist in the C++, but construction from vector is possible since
# a host_span can be implicitly constructed from a vector, so we add these for convenience
cdef device_buffer concatenate_masks "cudf::concatenate_masks"(
vector[column_view] views
) except +
cdef unique_ptr[column] concatenate_columns "cudf::concatenate"(
const vector[column_view] columns
vector[column_view] columns
) except +
cdef unique_ptr[table] concatenate_tables "cudf::concatenate"(
const vector[table_view] tables
vector[table_view] tables
) except +
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shwina these are the same as the original Cython code for concatenate, and the only ones currently used. So one idea is that I could revert my Cython changes in this PR. We could then wait and add host_span in Cython only if we need it... What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Cython definitions are nice to have. I suspect at some point we're going to need them, so let's not delete. Agree about keeping the concat definitions the same though.

Copy link
Contributor

@shwina shwina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cython LGTM!

@harrism
Copy link
Member Author

harrism commented Mar 24, 2021

@gpucibot merge

@rapids-bot rapids-bot bot merged commit b854598 into rapidsai:branch-0.19 Mar 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team 4 - Needs Review Waiting for reviewer to review or respond breaking Breaking change improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants