Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve String Last/First Storage Efficiency #12879

Merged
merged 1 commit into from
Sep 7, 2022

Commits on Sep 6, 2022

  1. Improve String Last/First Storage Efficiency

    -Add classes for writing cell values in LZ4 block compressed format.
    Payloads are indexed by element number for efficient random lookup
    -update SerializablePairLongStringComplexMetricSerde to use block
    compression
    -SerializablePairLongStringComplexMetricSerde also uses delta encoding
    of the Long by doing 2-pass encoding: buffers first to find min/max
    numbers and delta-encodes as integers if possible
    
    Entry points for doing block-compressed storage of byte[] payloads
    are the CellWriter and CellReader class. See
    SerializablePairLongStringComplexMetricSerde for how these are used
    along with how to do full column-based storage (delta encoding here)
    which includes 2-pass encoding to compute a column header
    rash67 committed Sep 6, 2022
    Configuration menu
    Copy the full SHA
    642aebc View commit details
    Browse the repository at this point in the history