Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve count vectorization: replace popcnt implementation with vector counting #4614

Merged
merged 27 commits into from
Apr 26, 2024
Merged

Improve count vectorization: replace popcnt implementation with vector counting #4614

merged 27 commits into from
Apr 26, 2024

Commits on Apr 21, 2024

  1. count vectorization: replace popcnt implementation with vector co… 

    …unting
    @AlexGuteniev
    AlexGuteniev committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    b1aacab View commit details
    Browse the repository at this point in the history

  2. Don't do extra reduce in the end

    @AlexGuteniev
    AlexGuteniev committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    6f134e8 View commit details
    Browse the repository at this point in the history

  3. As the SSE branch has no masked tail, can reduce in a single pace

    @AlexGuteniev
    AlexGuteniev committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    35d61ac View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2024

  1. Reduce as infrequently as possible

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    ce8d8a5 View commit details
    Browse the repository at this point in the history

  2. missing range coverage

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    af456db View commit details
    Browse the repository at this point in the history

  3. test counting zeros

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    c232f62 View commit details
    Browse the repository at this point in the history

  4. compare with expected in new coverage

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    498408c View commit details
    Browse the repository at this point in the history

  5. separate _Count_traits_N and reuse reduce

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    48efb24 View commit details
    Browse the repository at this point in the history

  6. formatting

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    403ef98 View commit details
    Browse the repository at this point in the history

  7. Stand away from overflows!

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    983e93a View commit details
    Browse the repository at this point in the history

  8. sizes are bytes

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    64b6fe6 View commit details
    Browse the repository at this point in the history

  9. counting overflow better coverage

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    85263d8 View commit details
    Browse the repository at this point in the history

  10. fewer ops to reduce

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    0a92efc View commit details
    Browse the repository at this point in the history

  11. Comments cleanup

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    d816df0 View commit details
    Browse the repository at this point in the history

  12. reduce 1-byte with sad instruction

    @AlexGuteniev
    AlexGuteniev committed Apr 22, 2024
    Configuration menu
    Copy the full SHA
    dedc0cc View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Simplify _Count_traits_8::_Reduce_avx() by reusing _Reduce_sse().

    @StephanTLavavej
    StephanTLavavej committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    015d4f7 View commit details
    Browse the repository at this point in the history

Commits on Apr 25, 2024

  1. Fix _Count_traits_4::_Max_count. 

    0x1FFF'FFFF would be strict for the SSE4.2 codepath, but
0xFFF'FFFF is strict for the AVX2 codepath.
    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7e39a04 View commit details
    Browse the repository at this point in the history

  2. Add detailed comments explaining each _Max_count.

    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    8d4aab5 View commit details
    Browse the repository at this point in the history

  3. For clarity, scope __m128i _Count_vector to each iteration of the S… 

    …SE4.2 loop.

It never carries information across iterations.
    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    50d610a View commit details
    Browse the repository at this point in the history

  4. For the AVX2 loop, scope __m256i _Count_vector separately for the m… 

    …ain loop and tail.

This allows us to avoid subtracting 1 from `_Max_count`, making it more similar to the SSE4.2 loop.
    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    c06d4ff View commit details
    Browse the repository at this point in the history

  5. Fix comment typo.

    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    3f95815 View commit details
    Browse the repository at this point in the history

  6. Change test_count_zero() to test more lengths.

    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    22b561b View commit details
    Browse the repository at this point in the history

  7. Restore popcnt approach

    @AlexGuteniev
    AlexGuteniev committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    e587698 View commit details
    Browse the repository at this point in the history

  8. Get my bounds back

    @AlexGuteniev
    AlexGuteniev committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    87749d2 View commit details
    Browse the repository at this point in the history

  9. typos

    @AlexGuteniev
    AlexGuteniev committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    2e1d6c7 View commit details
    Browse the repository at this point in the history

  10. restore SSE4.2 comment

    @AlexGuteniev
    AlexGuteniev committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    fc6dcbd View commit details
    Browse the repository at this point in the history

  11. Clarify comments.

    @StephanTLavavej
    StephanTLavavej committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    c5dd5c2 View commit details
    Browse the repository at this point in the history