Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-36036: [C++][Python][Parquet] Implement Float16 logical type #36073

Merged
merged 37 commits into from
Nov 15, 2023
Merged

GH-36036: [C++][Python][Parquet] Implement Float16 logical type #36073

merged 37 commits into from
Nov 15, 2023
+2,670 −1,211

Commits on Nov 13, 2023

  1. Regenerate thrift headers

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    3433217 View commit details
    Browse the repository at this point in the history

  2. Implement LogicalType class

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    0909cd1 View commit details
    Browse the repository at this point in the history

  3. Implement column statistics

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    3113573 View commit details
    Browse the repository at this point in the history

  4. Apply suggestion from code review 

    Co-authored-by: Antoine Pitrou <[email protected]>
    @benibus @pitrou
    benibus and pitrou committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    046e967 View commit details
    Browse the repository at this point in the history

  5. Add Float16 utils to Arrow

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    66efa36 View commit details
    Browse the repository at this point in the history

  6. Replace float_internal.h

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    e51d0d1 View commit details
    Browse the repository at this point in the history

  7. Minor test tweaks

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    a2f72ac View commit details
    Browse the repository at this point in the history

  8. Add tests for Float16 operators

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    1163b4e View commit details
    Browse the repository at this point in the history

  9. Support multiple endians in Float16 class

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    bc640ff View commit details
    Browse the repository at this point in the history

  10. Small refactor

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    2d7e65f View commit details
    Browse the repository at this point in the history

  11. Address more review points

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    5e925ac View commit details
    Browse the repository at this point in the history

  12. Support reading/writing arrow::HalfFloat

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    87d121c View commit details
    Browse the repository at this point in the history

  13. Fix MSVC truncation warning

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    a064bec View commit details
    Browse the repository at this point in the history

  14. Fix test input generation

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    6b3d61c View commit details
    Browse the repository at this point in the history

  15. Support conversions to/from float32

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    1758105 View commit details
    Browse the repository at this point in the history

  16. Remove Float16Base class

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    d41a0c5 View commit details
    Browse the repository at this point in the history

  17. Update/restructure comparison tests

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    aaef4b4 View commit details
    Browse the repository at this point in the history

  18. Fix comment

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    9e5cf14 View commit details
    Browse the repository at this point in the history

  19. Minor changes to Float16 class/tests

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    e124986 View commit details
    Browse the repository at this point in the history

  20. Update statistics and tests

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    a12176f View commit details
    Browse the repository at this point in the history

  21. Update Arrow reader

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    6496aef View commit details
    Browse the repository at this point in the history

  22. Remove big-endian handling in column writer

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    102dfb4 View commit details
    Browse the repository at this point in the history

  23. Tweak Arrow/Parquet schema tests

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    2a45f29 View commit details
    Browse the repository at this point in the history

  24. Support util::Float16 in random_real

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    d340a82 View commit details
    Browse the repository at this point in the history

  25. Update Arrow reader/writer tests

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    6285af1 View commit details
    Browse the repository at this point in the history

  26. Add To/FromDouble methods to Float16

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    554de9d View commit details
    Browse the repository at this point in the history

  27. Add tests for double conversions

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    40e58f5 View commit details
    Browse the repository at this point in the history

  28. Change misleading types

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    7407ce4 View commit details
    Browse the repository at this point in the history

  29. Some Float16 API changes 

    Reverted several prior changes that were accidentally pushed

Enabled construction from native floats

Removed `uint16_t` conversion operator since it doesn't behave
consistently with standard floats. As a result, rolled back some of the
prior changes to `random_real` used in the Parquet test utils
    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    bb4ca6a View commit details
    Browse the repository at this point in the history

  30. Refactor typed comparators

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    354f6f6 View commit details
    Browse the repository at this point in the history

  31. Add logical type to docs

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    ea5f5dc View commit details
    Browse the repository at this point in the history

  32. Replace public Float16(uint16_t) constructor

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    ab84630 View commit details
    Browse the repository at this point in the history

  33. Float16 tweaks, add constexpr tests

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    c8404bb View commit details
    Browse the repository at this point in the history

  34. Add test for ColumnIndex/BoundaryOrder

    @benibus
    benibus committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    157e0d7 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Tweak ToEndian methods

    @benibus
    benibus committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    5eb90d7 View commit details
    Browse the repository at this point in the history

  2. Relocate random Float16 function

    @benibus
    benibus committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    cb17f56 View commit details
    Browse the repository at this point in the history

  3. Add missing schema tests

    @benibus
    benibus committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    36b8a3b View commit details
    Browse the repository at this point in the history