-
Notifications
You must be signed in to change notification settings - Fork 90
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reorganization of evaluator code and renaming of modules #320
Conversation
in preparation for future updates to the mpi code, all evaluator code has been reorganized. All evaluators go into their respective module, and code sthared among 2 or more goes into a new util package. All relevant modules also have been renamed to future_{style of parallelization}
I will try to give it a look monday! |
I have a few questions:
|
I am, in parallel, working on expanding on the MPI code. I noticed that I could reuse parts of the multiprocessing code in this. However, it also gave rise to circular imports. To resolve these, at a minimum, I needed to move all reused parts into a separate util module. Moreover, for convenience, I prefer to have related functionality in the same module and keep the modules rather short. Therefore, I moved all evaluators to their respective modules (i.e., multiprocessing, mpi, ipyparallel). The net result is a code base that is more cleanly separated, easier to test in isolation, and prepares for further updates to mpi and multiprocressing. The prefix was ema, but that is not informative. I went with futures to indicate it has to do with parallization, but I am open to other suggestions. The reorganization resulted in import problems that caused the docs to fail. I fixed it quickly by making ipyparallel required. I looked at other solutions, but because I am subclassing stuff from ipyparallel, I could not use the method level imports as used with mpi4py. |
Thanks for the explanation. Why do we need a prefix at all? Edit: These are breaking changes right? As in it break current imports? |
If you don't, you can get namespace classes (e.g., No these are not breaking changes, because all relevant imports were available at the ema_workbench level already. It is only an internal reorganization. |
In preparation for future updates to the mpi code, all evaluator code has been reorganized. All evaluators go into their respective modules, and code shared among 2 or more goes into a new util package.
tests and docs have been updated as well to reflect these changes.