You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@WardLT made a loop to stop his code from submitting too many (100,000,000) tasks in one go to Parsl, because that uses too much memory. His code intermingles application specific app launching with load management.
There's some librification that could happen here: for example, user supplies a generator (for example a generator comprehension, or a for loop using yield) which provides a long series of tasks; some Parsl library function pulls tasks off that generator to keep Parsl loaded, but not too loaded (for example, by default 10x number of tasks in queues as workers are available).
@WardLT made a loop to stop his code from submitting too many (100,000,000) tasks in one go to Parsl, because that uses too much memory. His code intermingles application specific app launching with load management.
There's some librification that could happen here: for example, user supplies a generator (for example a generator comprehension, or a
for
loop usingyield
) which provides a long series of tasks; some Parsl library function pulls tasks off that generator to keep Parsl loaded, but not too loaded (for example, by default 10x number of tasks in queues as workers are available).@WardLT code for reference: https://github.com/HydrogenStorage/molecular-stability-computer/blob/main/compute_emin.py#L147
The text was updated successfully, but these errors were encountered: