Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make adios2_remote_server multithreaded and compress responses using … #4407

Merged
merged 2 commits into from
Dec 6, 2024

Conversation

pnorbert
Copy link
Contributor

…MGARD or ZFP (preference in this order) when an accuracy is set by user.

  • added "-t nThreads" option, default is 8
  • Use threads to compress responsesi and to return data to client. Adios Get() is still done in main thread because that is not thread-safe.

updates to Campaign engine:

  • Call PerformGets() in Campaign engine
  • Pass accuracy between CampaignReader and BP5 variables

update to Python API

  • added python bindings.Accuracy struct, and SetAccuracy()/GetAccuracy() to bindings, and variable.set_accuracy() and variable.get_accuracy() to python API.

Copy link
Contributor

@anagainaru anagainaru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot guarantee for all the threaded code, but I tested this and couldn't find a problem so far

…MGARD or ZFP (preference in this order) when an accuracy is set by user.

- added "-t nThreads" option, default is 8
- Use threads to compress responsesi and to return data to client. Adios Get() is still done in main thread because that is not thread-safe.

updates to Campaign engine:
- Call PerformGets() in Campaign engine
- Pass accuracy between CampaignReader and BP5 variables

update to Python API
- added python bindings.Accuracy struct, and SetAccuracy/GetAccuracy to bindings, and variable.set_accuracy() and variable.get_accuracy to python API.
@pnorbert
Copy link
Contributor Author

pnorbert commented Dec 6, 2024

Note that I can run the Remote.BPWriteMemorySelectionRead.GetRemote test fine on Frontier, so I will ignore the reported error on the OLCF CI and merge this now.

@pnorbert pnorbert merged commit e74f184 into ornladios:master Dec 6, 2024
38 of 39 checks passed
@pnorbert pnorbert deleted the remote_compression branch December 6, 2024 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants