Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace czlib with go-libdeflate #50

Merged
merged 4 commits into from
Dec 28, 2023
Merged

Conversation

oflebbe
Copy link
Contributor

@oflebbe oflebbe commented Sep 23, 2023

Performance can be improved by using libdeflate, which is better suited
zlib uncompression. Care was taken to reuse the buffer "data".

Since Workflow had errors I renovated it a bit. However I did not use
https://golangci-lint.run since it showed too many nitpicks.

Benchmark improvements from czlib to libdeflate:

benchmark                              old ns/op     new ns/op     delta
BenchmarkLondon-8                      296011237     275254714     -7.01%
BenchmarkLondon_withFiltersTrue-8      282195335     263935960     -6.47%
BenchmarkLondon_withFiltersFalse-8     231823251     200477972     -13.52%
BenchmarkLondon_nodes-8                216665255     180614979     -16.64%
BenchmarkLondon_ways-8                 167160662     140700970     -15.83%
BenchmarkLondon_relations-8            99489502      75263200      -24.35%

benchmark                              old allocs     new allocs     delta
BenchmarkLondon-8                      4811462        4808526        -0.06%
BenchmarkLondon_withFiltersTrue-8      4811463        4808515        -0.06%
BenchmarkLondon_withFiltersFalse-8     1367676        1364724        -0.22%
BenchmarkLondon_nodes-8                3398498        3395559        -0.09%
BenchmarkLondon_ways-8                 1799045        1796099        -0.16%
BenchmarkLondon_relations-8            463103         460152         -0.64%

benchmark                              old bytes     new bytes     delta
BenchmarkLondon-8                      938187018     924789892     -1.43%
BenchmarkLondon_withFiltersTrue-8      938186796     924787588     -1.43%
BenchmarkLondon_withFiltersFalse-8     379851201     366452840     -3.53%
BenchmarkLondon_nodes-8                632788308     619391213     -2.12%
BenchmarkLondon_ways-8                 451758725     438360054     -2.97%
BenchmarkLondon_relations-8            198025084     184626277     -6.77%```

@KacperPerschke
Copy link

Why not compress/zlib?

@paulmach
Copy link
Owner

Why not compress/zlib?

Speed. See the comment in the osmpdb/README.md. 25% faster with the current lib. This change looks to make it even faster.

@paulmach paulmach changed the base branch from master to deflate December 28, 2023 06:04
@paulmach paulmach merged commit 82384b5 into paulmach:deflate Dec 28, 2023
2 of 3 checks passed
@paulmach
Copy link
Owner

I merged these change into #55 so that I could fix the issues. I was unable to run locally due to some linker issues, will need to debug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants