Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Snapshot creation is inefficient #6225

Closed
gibson042 opened this issue Sep 16, 2022 · 0 comments · Fixed by #6248
Closed

Snapshot creation is inefficient #6225

gibson042 opened this issue Sep 16, 2022 · 0 comments · Fixed by #6248
Assignees
@gibson042
Labels
enhancement New feature or request performance Performance related issues SwingSet package: SwingSet
Milestone
Mainnet 1 RC0

Comments

@gibson042
Copy link
Member

What is the Problem Being Solved?

#6188 revealed some inefficiencies in snapshot creation:

  • Use of a potentially-remote file system for temporary files
  • Multiple serial reads of raw snapshot data

Description of the Design

We should place temporary files in TMPDIR, and tee the snapshot data to parallel readers.

Security Considerations

None expected

Test Plan

Unit tests will verify functionality, but the performance changes theirselves should be evaluated on a real network.
@gibson042 gibson042 added enhancement New feature or request SwingSet package: SwingSet performance Performance related issues labels Sep 16, 2022
@Tartuffo Tartuffo added this to the Mainnet 1 RC0 milestone Sep 16, 2022
@dckc dckc mentioned this issue Sep 16, 2022
Closed
gibson042 added a commit that referenced this issue Sep 16, 2022
@gibson042
perf(swing-store): Improve the efficiency of writing snapshots
ae72eb2 
* Use TMPDIR for temporary files.
* Parallelize reading raw snapshot data for hash computation and compression.

Fixes #6225
@gibson042 gibson042 mentioned this issue Sep 16, 2022
Merged
mhofman pushed a commit that referenced this issue Sep 16, 2022
@gibson042 @mhofman
perf(swing-store): Improve the efficiency of writing snapshots
236d3bd 
* Use TMPDIR for temporary files.
* Parallelize reading raw snapshot data for hash computation and compression.

Fixes #6225
mhofman pushed a commit that referenced this issue Sep 17, 2022
@gibson042 @mhofman
perf(swing-store): Improve the efficiency of writing snapshots
39a0e3d 
* Use TMPDIR for temporary files.
* Parallelize reading raw snapshot data for hash computation and compression.

Fixes #6225
turadg pushed a commit that referenced this issue Sep 17, 2022
@gibson042 @mergify
perf(swing-store): Improve the efficiency of writing snapshots
ef78e7d 
* Use TMPDIR for temporary files.
* Parallelize reading raw snapshot data for hash computation and compression.

Fixes #6225
@mergify mergify bot closed this as completed in #6248 Sep 17, 2022
@mhofman mhofman mentioned this issue Sep 30, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gibson042 gibson042

Labels
enhancement New feature or request performance Performance related issues SwingSet package: SwingSet
Projects
None yet
Milestone
Mainnet 1 RC0
Development

Successfully merging a pull request may close this issue.

Improve the efficiency of writing snapshots Agoric/agoric-sdk
2 participants
@Tartuffo @gibson042