Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewriting row/column conversions for Spark <-> cudf data conversions #8444

Merged

Commits on Sep 30, 2021

  1. Configuration menu
    Copy the full SHA
    cf58b89 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6e869b6 View commit details
    Browse the repository at this point in the history
  3. Updates and bug fixing

    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    2703baf View commit details
    Browse the repository at this point in the history
  4. Updating windows to be generated in a square way so we can have more …

    …data to write out as 8-byte writes from shared memory.
    
    Shuffled some of the copy to GPU code up so it can start the copy sooner and hopefully won't force stalls.
    Some bug fixes.
    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    63a6636 View commit details
    Browse the repository at this point in the history
  5. Adding row to column conversion code. Performance falls off a cliff, …

    …but starts out reasonably. I haven't looked at this in nsight yet.
    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    b444279 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    fbe5dd5 View commit details
    Browse the repository at this point in the history
  7. updating conversion code. Found out bit operations are on 32-bit valu…

    …es, so they can't be used since row data has byte-aligned validity. Performance improvements on the row to column side.
    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    f8bc01f View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    636b235 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    3bff2aa View commit details
    Browse the repository at this point in the history
  10. Updates and bug fixing

    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    8e52ba1 View commit details
    Browse the repository at this point in the history
  11. Updating windows to be generated in a square way so we can have more …

    …data to write out as 8-byte writes from shared memory.
    
    Shuffled some of the copy to GPU code up so it can start the copy sooner and hopefully won't force stalls.
    Some bug fixes.
    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    fdfcb39 View commit details
    Browse the repository at this point in the history
  12. Adding row to column conversion code. Performance falls off a cliff, …

    …but starts out reasonably. I haven't looked at this in nsight yet.
    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    5cf1cf1 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    9af3258 View commit details
    Browse the repository at this point in the history
  14. updating conversion code. Found out bit operations are on 32-bit valu…

    …es, so they can't be used since row data has byte-aligned validity. Performance improvements on the row to column side.
    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    1d0245b View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    65490e0 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    2a57ce6 View commit details
    Browse the repository at this point in the history
  17. Updates and bug fixes

    hyperbolic2346 committed Sep 30, 2021
    Configuration menu
    Copy the full SHA
    ed5492e View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2021

  1. Configuration menu
    Copy the full SHA
    158a864 View commit details
    Browse the repository at this point in the history
  2. Fixing merge issue

    hyperbolic2346 committed Oct 1, 2021
    Configuration menu
    Copy the full SHA
    02cb81b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    bae16f6 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2021

  1. Configuration menu
    Copy the full SHA
    3656848 View commit details
    Browse the repository at this point in the history
  2. Use the new row<->col method

    Added a new method `convertFromRowsFixedWidthOptimized`
    and `convertToRowsFixedWidthOptimized` to be used for when columns
    are < 100. Otherwise use the new method
    
    This is currently failing simple tests
    razajafri committed Oct 6, 2021
    Configuration menu
    Copy the full SHA
    76e4099 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2021

  1. Configuration menu
    Copy the full SHA
    966c34c View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mwilson/row_conversion' of github.com:hyperbolic2346/cu…

    …df into mwilson/row_conversion
    hyperbolic2346 committed Oct 7, 2021
    Configuration menu
    Copy the full SHA
    aeaf28c View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2021

  1. Configuration menu
    Copy the full SHA
    6452e8e View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2021

  1. Configuration menu
    Copy the full SHA
    c0e9895 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2021

  1. Configuration menu
    Copy the full SHA
    b9df725 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8d00447 View commit details
    Browse the repository at this point in the history
  3. fixed typo

    razajafri committed Oct 21, 2021
    Configuration menu
    Copy the full SHA
    c4b0242 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2021

  1. Merge branch 'mwilson/row_conversion' of github.com:hyperbolic2346/cu…

    …df into mwilson/row_conversion
    hyperbolic2346 committed Oct 22, 2021
    Configuration menu
    Copy the full SHA
    bc4ed9b View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2021

  1. Updating for actual PR. Fixed a few last minute bugs, removed cudf-la…

    …nd code that was there for testing and benchmarking.
    hyperbolic2346 committed Oct 23, 2021
    Configuration menu
    Copy the full SHA
    e92989c View commit details
    Browse the repository at this point in the history
  2. removing unused header, suppressing shared warning for barrier, updat…

    …ing java bindings to use the correct namespace
    hyperbolic2346 committed Oct 23, 2021
    Configuration menu
    Copy the full SHA
    131ca58 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2021

  1. Configuration menu
    Copy the full SHA
    a74c01c View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2021

  1. Configuration menu
    Copy the full SHA
    d013e8b View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2021

  1. Configuration menu
    Copy the full SHA
    70e39cd View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2021

  1. Configuration menu
    Copy the full SHA
    a10375c View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2021

  1. Configuration menu
    Copy the full SHA
    64c8374 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2021

  1. addressed review concerns

    razajafri committed Nov 15, 2021
    Configuration menu
    Copy the full SHA
    f8ea2b1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c88472a View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2021

  1. Configuration menu
    Copy the full SHA
    00e58d7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b9f42cd View commit details
    Browse the repository at this point in the history
  3. Updates and bug fixing

    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    6a267ab View commit details
    Browse the repository at this point in the history
  4. Updating windows to be generated in a square way so we can have more …

    …data to write out as 8-byte writes from shared memory.
    
    Shuffled some of the copy to GPU code up so it can start the copy sooner and hopefully won't force stalls.
    Some bug fixes.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    170a771 View commit details
    Browse the repository at this point in the history
  5. Adding row to column conversion code. Performance falls off a cliff, …

    …but starts out reasonably. I haven't looked at this in nsight yet.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    a82cee8 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    31f3b4a View commit details
    Browse the repository at this point in the history
  7. updating conversion code. Found out bit operations are on 32-bit valu…

    …es, so they can't be used since row data has byte-aligned validity. Performance improvements on the row to column side.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    b044f8b View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    d2a33ed View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    7bcf41c View commit details
    Browse the repository at this point in the history
  10. Updates and bug fixing

    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    17f1e5d View commit details
    Browse the repository at this point in the history
  11. Updating windows to be generated in a square way so we can have more …

    …data to write out as 8-byte writes from shared memory.
    
    Shuffled some of the copy to GPU code up so it can start the copy sooner and hopefully won't force stalls.
    Some bug fixes.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    dfda0f3 View commit details
    Browse the repository at this point in the history
  12. Adding row to column conversion code. Performance falls off a cliff, …

    …but starts out reasonably. I haven't looked at this in nsight yet.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    5c0e52c View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    cdd02d0 View commit details
    Browse the repository at this point in the history
  14. updating conversion code. Found out bit operations are on 32-bit valu…

    …es, so they can't be used since row data has byte-aligned validity. Performance improvements on the row to column side.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    7bb0496 View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    2b069ca View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    92f52cd View commit details
    Browse the repository at this point in the history
  17. Updates and bug fixes

    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    83118d2 View commit details
    Browse the repository at this point in the history
  18. Fixing merge issue

    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    d563eaa View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    5b6688d View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    53912ca View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    698817a View commit details
    Browse the repository at this point in the history
  22. Use the new row<->col method

    Added a new method `convertFromRowsFixedWidthOptimized`
    and `convertToRowsFixedWidthOptimized` to be used for when columns
    are < 100. Otherwise use the new method
    
    This is currently failing simple tests
    razajafri authored and hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    fb6dd51 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    b0173bf View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    81cbaa6 View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    58eb43f View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    06837f0 View commit details
    Browse the repository at this point in the history
  27. fixed typo

    razajafri authored and hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    2c4e12f View commit details
    Browse the repository at this point in the history
  28. Updating for actual PR. Fixed a few last minute bugs, removed cudf-la…

    …nd code that was there for testing and benchmarking.
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    fa4f0d3 View commit details
    Browse the repository at this point in the history
  29. removing unused header, suppressing shared warning for barrier, updat…

    …ing java bindings to use the correct namespace
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    e9938b9 View commit details
    Browse the repository at this point in the history
  30. Configuration menu
    Copy the full SHA
    3c6b1e5 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    630222a View commit details
    Browse the repository at this point in the history
  32. Configuration menu
    Copy the full SHA
    5e66d7c View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    37feaa1 View commit details
    Browse the repository at this point in the history
  34. Merge branch 'mwilson/row_conversion' of github.com:hyperbolic2346/cu…

    …df into mwilson/row_conversion
    hyperbolic2346 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    e09ab04 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2021

  1. Configuration menu
    Copy the full SHA
    27d44d9 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2021

  1. Configuration menu
    Copy the full SHA
    3a48844 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2021

  1. Configuration menu
    Copy the full SHA
    2844240 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2021

  1. Configuration menu
    Copy the full SHA
    9770def View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2021

  1. Configuration menu
    Copy the full SHA
    7595eaf View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2021

  1. Configuration menu
    Copy the full SHA
    74afad7 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2021

  1. Configuration menu
    Copy the full SHA
    652fc33 View commit details
    Browse the repository at this point in the history

Commits on Dec 23, 2021

  1. Configuration menu
    Copy the full SHA
    57a84e4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7fbe10d View commit details
    Browse the repository at this point in the history

Commits on Jan 4, 2022

  1. Configuration menu
    Copy the full SHA
    d47360d View commit details
    Browse the repository at this point in the history
  2. Updating namespace

    hyperbolic2346 committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    9b50271 View commit details
    Browse the repository at this point in the history
  3. updating namespace

    hyperbolic2346 committed Jan 4, 2022
    Configuration menu
    Copy the full SHA
    fb7566c View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2022

  1. Configuration menu
    Copy the full SHA
    5e1cf97 View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2022

  1. Configuration menu
    Copy the full SHA
    a1e3545 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'mwilson/row_conversion' of github.com:hyperbolic2346/cu…

    …df into mwilson/row_conversion
    hyperbolic2346 committed Jan 7, 2022
    Configuration menu
    Copy the full SHA
    8ef3bbe View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2022

  1. Configuration menu
    Copy the full SHA
    4c750a9 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0d0015a View commit details
    Browse the repository at this point in the history