Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compress the ECC sets by default #488

Closed
aborgna-q opened this issue Jul 16, 2024 · 1 comment · Fixed by #498
Closed

Compress the ECC sets by default #488

aborgna-q opened this issue Jul 16, 2024 · 1 comment · Fixed by #498
Assignees
@aborgna-q

Comments

@aborgna-q
Copy link
Collaborator

aborgna-q commented Jul 16, 2024 •
edited
Loading

We should use something like zstd to compress/decompress the rewrite sets, instead of storing the full files.

The included nam_6_3.rwr is 70MB, but compresses with zstd -9 to 4.4MB.

Note that the file already gets compressed when packaging the python wheels, so #486 is still necessary.
@aborgna-q
Copy link
Collaborator Author

Related to CQCL/hugr#1209, but we may have to compress the .rwr directly from our side.

@aborgna-q aborgna-q mentioned this issue Jul 19, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Jul 22, 2024
@aborgna-q
feat!: Compress binary ECCs using zlib (#498)
d9a713c 
Breaking change for the compiled `.rwr` ECC sets. The format is now
compressed using `zstd`.

Comparative figures for the `nam_6_3.rwr` included in the python
package:

- File size reduced from `69MB` to `4.4MB`.
- Load time on my machine reduced from `~560ms` to `~420ms`.
- (offline) ECC compilation time increased from `2.4s` to `3s`.
This is using the highest compression rate. Faster encodings produce
files of around `5MB`.

Closes #488

BREAKING CHANGE: `.rwr` ECC files generated with older versions are no
longer supported. Please recompile them, or compress the file with
`zstd`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aborgna-q aborgna-q

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat!: Compress binary ECCs using zlib CQCL/tket2
1 participant
@aborgna-q