recursive @threads blocks unsupported

[jean-pierreBoth]

The problem occurs when a function having a threaded loop calling another function with
a threaded loop inside.

here is a code sample computing the mean of a random matrix.
expected result 0.5

debuglock=Threads.RecursiveSpinLock()

using Distances

n=100

A=rand(n,n)

vmat=Array{Array{Float64,2},1}(2)
vmat[1]=rand(n,n)
vmat[2]=rand(n,n)



#compute sum of column i
function fbidon(A::Array{Float64,2}, i::Int64, D::Distances.SemiMetric)
    lock(debuglock)
    @printf STDOUT "\n  fbidon thread %d  " Threads.threadid()
    unlock(debuglock)
    # to get cpu doing something
    evaluate(D, A[:,i], A[:,1])
    #
    v=sum(A[:,i])
    #
    lock(debuglock)
    @printf STDOUT "\n  fbidon thread %d  end " Threads.threadid()
    unlock(debuglock)
    #
    return v
end


function meanMatThread(A::Array{Float64,2})
    lock(debuglock)
    @printf STDOUT "\n meanMatThreadedF  thread %d  " Threads.threadid()
    unlock(debuglock)
    #
    n=size(A)[2]
    v=zeros(n)
    D=Jaccard()
    vid=Array{Int64,1}(n)
    Threads.@threads for i in 1:n
        vid[i] = Threads.threadid()
        v[i] = fbidon(A,i,D)
    end
    lock(debuglock)
    @printf STDOUT "\n meanMatThreadedF  thread %d  end " Threads.threadid()
    unlock(debuglock)
    sum(v)/(n*n)
end


function meanMatThreadThread(vm::Array{Array{Float64,2},1}) 
    res=Array{Float64,1}(2)
    Threads.@threads for i in 1:2
        res[i] = meanMatThread(vm[i])
    end
    return res
end

the result in detail is given in file bugThread.txt
summary :
call to meanMat(vmat[1]) OK 0.49068859745047
call to meanMat(vmat[2]) OK 0.5044064092497642

call to res=meanMatThreadThread(vmat)
result is false
2-element Array{Float64,1}:
0.124943
0.128193

I ran with 4 threads, and I get the result of 1/4 threads.
With n = 50000 it I get really 0.125 !!

Moreover the interpreter gets in a fuzzy state as described in details in bugThread.txt.

My configuration

julia> versioninfo(true)
Julia Version 0.5.0-rc3+0
Commit e6f843b (2016-08-22 23:43 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
  WORD_SIZE: 64
           Debian GNU/Linux testing (stretch)
  uname: Linux 4.6.0-1-amd64 #1 SMP Debian 4.6.4-1 (2016-07-18) x86_64 unknown
Memory: 125.84801864624023 GB (108943.20703125 MB free)
Uptime: 357928.0 sec
Load Avg:  0.0703125  0.17724609375  0.1572265625
Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz: 
          speed         user         nice          sys         idle          irq
#1-32  1200 MHz    5257848 s    6463890 s    1669061 s  1131089210 s          0 s

  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

Environment:

  MANPATH = 
  TERM = xterm-256color
  LD_LIBRARY_PATH = /usr/local/qwt-6.1.2/lib:/usr/local/lib:/home.2/jpboth/Spectro/Spectviewer/lib/x86_64:/home.2/jpboth/lib:/usr/local/qwt-6.1.2/lib:/usr/local/lib:/home.2/jpboth/Spectro/Spectviewer/lib/x86_64:/home.2/jpboth/lib:
  JULIA_NUM_THREADS = 4
  PATH = /home.1/Opt/julia-0.5.0.rc3/bin:/home.2/jpboth/Spectro/Spectviewer/bin/x86_64:/home.1/Opt/julia-0.5.0.rc3/bin:/home.2/jpboth/Spectro/Spectviewer/bin/x86_64:/home.1/Opt/julia/bin:/home.2/jpboth/Spectro/Spectviewer/bin/x86_64:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
  SPECTVIEWER_HOME = /home.2/jpboth/Spectro/Spectviewer
  HOME = /home.2/jpboth
  CLASSPATH = /usr/local/jdk1.8.0_45/jre/bin:/home.2/jpboth/lib/jclasses
  INFOPATH = /usr/share/info:/usr/local/info:/usr/local/share/info:
  WINDOWPATH = 1
  JAVAPATH = /usr/local/jdk1.8.0_45/bi

bugThread.txt

[yuyichao]

I don't think we support nested @threads.

[jean-pierreBoth]

I understand it is a bit messy to have thread safe nested macros.
But if I use someone else function without knowing it is threaded? Or the reverse
if I export a threaded function for my uers?
how can I be sure too avoid the situation?

[yuyichao]

I think we should have the feature so this should be fixed. Just stating that our really simple runtime probably doesn't handle this and is probably why you see the weird result.

[jean-pierreBoth]

I know multi threading is a real headache...
But it would be really nice to have this to complement your nice task formalism.
I hope you you will find a solution.

Thank you for your work with Julia!

[ViralBShah]

Yes, we need to leave this open.

[oxinabox]

@threads is just another of the many many nonthread safe operations.

Solving it right now for the very limitted @threads macro -- where there is no option other than to run something on all threads, and there is no way to run something on a single thread, is easy --- just at a Atomic (or lock protected) check and set asking if we are currently in a @threads block, and if so, then to act serially.

But this becomes harder when more thread functionality is added; I think

[StefanKarpinski]

This will definitely work once threading is less experimental, but it will take some time and effort.