Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Class attributes for parallelization of simulation APIs which require a license check #57

Open
FWuellhorst opened this issue Aug 21, 2022 · 0 comments
Assignees
Labels
persistent Possibly won't be solved

Comments

@FWuellhorst
Copy link
Collaborator

Currently, we use dictionaries, e.g. fmu_instances to enable the process of setting up the simulation APIs, checking a possible license server only n_cpu times, and then simulate again using multiprocessings pool.

The only way I found to enable this setup was to use class attributes, instead of instance attributes. Class attributes are kept in memory, while the pickle creates a new instance each time pool.map (or apply async o.s.) is called.

Researching how multiprocessing and pickle packages work, I could not come up with a better solution.
On the single core, an instance attribute is used. If we are going to multiple cores, the API is stored in the class attribute.
The main issue with using class attributes is that it introduces possible bugs when users implement nested multiprocessing setups.
However, as the time-consuming part is mostly on the simulation part, such nested setups should not be necessary anyway.

@FWuellhorst FWuellhorst added the persistent Possibly won't be solved label Aug 21, 2022
@FWuellhorst FWuellhorst self-assigned this Aug 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
persistent Possibly won't be solved
Projects
None yet
Development

No branches or pull requests

1 participant