Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implicitly wrap comprehensions in let for each capture global #5843

Closed
StefanKarpinski opened this issue Feb 18, 2014 · 2 comments
Closed

implicitly wrap comprehensions in let for each capture global #5843

StefanKarpinski opened this issue Feb 18, 2014 · 2 comments
Labels
breaking This change will break code parallelism Parallel or distributed computation

Comments

@StefanKarpinski
Copy link
Member

The proposal is that when you write, in global context, something like this:

[ f(x,α,β,γ) for x in itr ]

This actually means this:

let f = f, α = α, β = β, γ = γ
  [ f(x,α,β,γ) for x in itr ]
end

The idea is based on the theory that the evaluations of comprehension expressions are independent and implicitly concurrent. Thus, even if the bindings of the global variables f, α, β and γ are changed by the evaluation of any of the comprehension expressions, that change shouldn't be visible until after the evaluation of the entire comprehension. See #1571, wherein each instance of the comprehension variables were made indepdendent, as part of this theory. It has also been proposed that comprehensions could actually use multiple threads and be evaluated in parallel. This change would make their semantics more fully concurrent, thereby facilitating the possibility of actually evaluating comprehensions in parallel. Aside from increased concurrency, an immediate and significant benefit of this change would be to drastically improve the type inference of comprehensions in global scope, which has been a very commonly raised issue (see this thread, for example).

@JeffBezanson
Copy link
Member

See also #524.

It is interesting to note that we already do a transformation like this for parallel constructs like @spawn and @async. The idea there was that all used variables are captured immediately to be sent somewhere else for evaluation. Then we do it locally too, to give more similar behavior for remote and local spawns.

@JeffBezanson
Copy link
Member

I believe this is superseded by #7258

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking This change will break code parallelism Parallel or distributed computation
Projects
None yet
Development

No branches or pull requests

2 participants