Use stronger compression in eval #509
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrews05
force-pushed
the
evalcompression
branch
from
c1c1d9e to
09e0766
Compare
andrews05
force-pushed
the
evalcompression
branch
from
09e0766 to
2ea12fc
Compare
shssoichiro
approved these changes
Jul 8, 2023
shssoichiro
pushed a commit
that referenced
this pull request
Jul 11, 2023
This adds a new palette sorting algorithm that attempts to minimise entropy by an approximate solution to the Traveling Salesman Problem. The algorithm comes from "An efficient Re-indexing algorithm for color-mapped images" by Battiato et al (https://ieeexplore.ieee.org/document/1344033). It's fast and effective and works in addition to the luma sort (which remains the single most effective sort). In order to keep lower presets fast though, I've only enabled this for o3 and higher. Results on a set of 190 indexed images at `-o5`: 18,932,727 bytes - master 18,578,306 bytes - PR 18,559,863 bytes - PR + #509 (These images may be particularly suited to alternative sorting methods - the gains here are not necessarily what should be expected on average) Note I looked into the 120 different palette sorting methods from TruePNG, as mentioned in #74 (and seen in action in the Zopfli KrzYmod fork). They're... largely ineffective. The combination of all 120 methods are outperformed by just the existing luma sort plus this new one. That's not to say there's nothing further to be gained from them, but trying to brute force all the combinations definitely seems like a bad idea. There are other algorithms I hope to explore in future... @ace-dent Thought this might interest you UPDATE: I realised a quick tweak to alpha values in the luma sort can provide a great improvement on images with transparency. The following numbers were taken with PR #509 as base. `-o2`: 19,065,549 bytes - base (luma sort) 18,949,747 bytes - modified luma sort `-o5`: 18,922,165 bytes - base (luma sort) 18,559,863 bytes - new sorting algorithm + luma sort 18,544,813 bytes - new sorting algorithm + modified luma sort
Pr0methean
pushed a commit
to Pr0methean/oxipng
that referenced
this pull request
Dec 1, 2023
Pr0methean
pushed a commit
to Pr0methean/oxipng
that referenced
this pull request
Dec 1, 2023
This adds a new palette sorting algorithm that attempts to minimise entropy by an approximate solution to the Traveling Salesman Problem. The algorithm comes from "An efficient Re-indexing algorithm for color-mapped images" by Battiato et al (https://ieeexplore.ieee.org/document/1344033). It's fast and effective and works in addition to the luma sort (which remains the single most effective sort). In order to keep lower presets fast though, I've only enabled this for o3 and higher. Results on a set of 190 indexed images at `-o5`: 18,932,727 bytes - master 18,578,306 bytes - PR 18,559,863 bytes - PR + shssoichiro#509 (These images may be particularly suited to alternative sorting methods - the gains here are not necessarily what should be expected on average) Note I looked into the 120 different palette sorting methods from TruePNG, as mentioned in shssoichiro#74 (and seen in action in the Zopfli KrzYmod fork). They're... largely ineffective. The combination of all 120 methods are outperformed by just the existing luma sort plus this new one. That's not to say there's nothing further to be gained from them, but trying to brute force all the combinations definitely seems like a bad idea. There are other algorithms I hope to explore in future... @ace-dent Thought this might interest you UPDATE: I realised a quick tweak to alpha values in the luma sort can provide a great improvement on images with transparency. The following numbers were taken with PR shssoichiro#509 as base. `-o2`: 19,065,549 bytes - base (luma sort) 18,949,747 bytes - modified luma sort `-o5`: 18,922,165 bytes - base (luma sort) 18,559,863 bytes - new sorting algorithm + luma sort 18,544,813 bytes - new sorting algorithm + modified luma sort
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes the compression level used in the evaluator to be a little stronger.
-o0remains at level 5, matching the main zc level-o1and-o2(or anytime--fastis enabled) use level 7-o3and higher use level 8-Zuses level 8, regardless of other optionsRecent changes to reductions and being more picky about exactly which ones are evaluated means that we can see a (very) minor compression improvement from this without sacrificing any performance.
Small benchmark on 50 files, 2,143,739 bytes. The loss from
--fastwith zopfli drops from 0.31% to 0.11%.-o5 -Z --fastmaster: 1,456,118 bytes-o5 -Zmaster: 1,451,546 bytes-o5 -Z --fastPR: 1,452,726 bytes-o5 -ZPR: 1,451,124 bytes