pmap memory leak

[denizyuret]

The code below leaks in Version 0.3.7 (2015-03-23 21:36 UTC) and Version 0.4.0-dev+3718 (2015-03-07 21:17 UTC). gc() does not seem to help. Replacing pmap with map stops the leak.

function main()
    for epoch=1:100
        @show epoch
        a = helper()
        @everywhere gc()
        @everywhere println(Base.PGRP.refs)
    end
end

function helper()
    @time p = pmap(workers()) do w
        x = rand(Float32, 1<<10, 1<<15)
    end
    return 0
end

main()

[timholy]

#3934, #8912, #6597

[amitmurthy]

Thanks to @vtjnash 's fix 3bbc5fc

memory usage reaches a steady high state quite soon and does not keep growing when tested with the following code (julia -p 2):

function main()
    for epoch=1:100
        @show epoch
        a = helper()
        @everywhere gc()
    end
end

function helper()
    @time p = pmap(workers()) do w
        x = ones(10^8)
    end
    return 0
end

main()

The resident memory (after 100 loops) for the master is around 4.5G and for the workers around 800 MB each. That is probably a different issue.

Closing this issue.