Julia binary goes in infinite wait on Ubuntu 14.10 (ppc64le)

[MaheshWaidande]

Hi All,

I am working on building/porting Julia on ppc64le architecture. I am using Ubuntu 14.10 on top of ppc64le hardware. I am able to build/compile Julia master branch on it Ubuntu. When I try to execute julia binary, it goes into indefinite wait and I am not able to view julia prompt.

Any pointer on this strange behavior would be great help.

Below is some background information which will help to understand/ resolve a problem.

a). Machine details :

test@pts00433-vm1:~/Mahesh/julia/julia$ uname -a
Linux pts00433-vm1 3.16.0-30-generic #40-Ubuntu SMP Mon Jan 12 22:07:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux

b). Content of Make.user file

include /home/test/Mahesh/julia/julia/Make.powerpc

CFLAGS += -D__ppc64__
CXXFLAGS += -D__ppc64__
LDFLAGS = -llzma
VERBOSE = 1
test@pts00433-vm1:~/Mahesh/julia/julia$

c). Content of Make.powerpc file

#This Make.ppc script should help you get started building Julia on a PowerPC machine
#$ echo "include $(JULIAHOME)/Make.powerpc" >> Make.user

#most of our dependencies these contain m4 scripts that are too old (specifically libtool.m4) to know about ppc64le
#so for now, we must use the system ones
USE_SYSTEM_FFTW=1
USE_SYSTEM_PCRE=0
USE_SYSTEM_ARPACK=1
USE_SYSTEM_GMP=1
USE_SYSTEM_MPFR=1

#openblas doesn't know about ld version 2 assembly PROLOGUE syntax
USE_SYSTEM_BLAS=1

#it's not optimized, and we trust the system libm
USE_SYSTEM_LIBM=1

#this one doesn't seem to be able to compile for assembly code
USE_SYSTEM_LIBUNWIND=1

#this one takes awhile to build, but it's generally worthwhile -- your choice
USE_SYSTEM_LLVM?=0
ifeq ($(USE_SYSTEM_LLVM),1)
LLVM_CONFIG=llvm-config-3.5
else
LLVM_ASSERTIONS=1
LLVM_VER=3.6.0
LLVM_ASSERTIONS=1
endif

#On a newly allocated machine, you'll need the following dependencies:
#build essentials:
#$ sudo apt-get install git g++ gfortran make cmake libedit-dev zlib1g-dev libssl-dev
#system lib replacements:
#$ sudo apt-get install libfftw3-dev libunwind8-dev libllvm3.5 libblas3 libarpack2-dev libedit-dev libpcre3-dev libgmp-dev libmpfr-dev
#autogen essentials:
#$ sudo apt-get install libtool autoconf
test@pts00433-vm1:~/Mahesh/julia/julia$

d). While compiling below code snippet I was getting segmentation fault. For resolving segmentation fault I made changes
in options.h file, I turn on MEMDEBUG flag in options.h file.

julia/usr/bin/julia -C native --output-o /home/test/Mahesh/julia/julia/usr/lib/julia/inference0.o -f coreimg.jl

diff of options.h file,

diff --git a/src/options.h b/src/options.h
index 23540ae..e2318cb 100644
--- a/src/options.h
+++ b/src/options.h
@@ -40,7 +40,7 @@
 // with MEMDEBUG, every object is allocated explicitly with malloc, and
 // filled with 0xbb before being freed. this helps tools like valgrind
 // catch invalid accesses.
-// #define MEMDEBUG
+#define MEMDEBUG

 // GC_VERIFY force a full verification gc along with every quick gc to ensure no
 // reachable memory is freed

e). I attached gdb to Julia and try to find out the reason why I am not getting Julia command prompt.
Below is stacktrace of gdb.

test@pts00433-vm1:~/Mahesh/julia/julia$ gdb ./julia
(gdb) r
Starting program: /home/test/Mahesh/julia/julia/julia
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+5769 (2015-07-03 04:36 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 2c1ac18* (0 days old master)
|__/                   |  powerpc64le-linux-gnu

^C
Program received signal SIGINT, Interrupt.
0x00003fffb6a138fc in __pause_nocancel () at ../sysdeps/unix/syscall-template.S:81
81      ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) bt
#0  0x00003fffb6a138fc in __pause_nocancel () at ../sysdeps/unix/syscall-template.S:81
#1  0x00003fffb4f5fb10 in julia_wait_18019 () at task.jl:302
#2  0x00003fffb4f69bc4 in julia_wait_18197 (c=<optimized out>) at task.jl:225
#3  0x00003fffb6c00330 in jl_apply (nargs=1, args=0x3fffffffef38, f=0x12c5e370) at julia.h:1260
#4  jl_trampoline (F=0x12c5e370, args=0x3fffffffef38, nargs=<optimized out>) at builtins.c:975
#5  0x00003fffb6bf1b00 in jl_apply (nargs=1, args=0x3fffffffef38, f=<optimized out>) at julia.h:1260
#6  jl_apply_generic (F=0x12c59b00, args=0x3fffffffef38, nargs=<optimized out>) at gf.c:1632
#7  0x00003fffb2c60088 in julia_anonymous_24015 ()
#8  0x00003fffb6c00330 in jl_apply (nargs=0, args=0x0, f=0x12a77d30) at julia.h:1260
#9  jl_trampoline (F=0x12a77d30, args=0x0, nargs=<optimized out>) at builtins.c:975
#10 0x00003fffb6b04640 in jl_apply (nargs=0, args=0x0, f=<optimized out>) at julia.h:1260
#11 start_task () at task.c:232
#12 0x00003fffb6c66f10 in set_base_ctx (__stk=<optimized out>) at task.c:241
#13 0x0000000000000000 in ?? ()
(gdb)

*** Please observe above frame 2 and 3 and which will clearly shows that ./julia binary is in infinite wait.

Here is code snippet of task.jl file,

test@pts00433-vm1:~/Mahesh/julia/julia$ cat -n ./base/task.jl | head -n 234 | tail -18
   217
   218  function wait(c::Condition)
   219      ct = current_task()
   220
   221      ct.state = :waiting
   222      push!(c.waitq, ct)
   223
   224      try
   225          return wait()
   226      catch
   227          filter!(x->x!==ct, c.waitq)
   228          if ct.state == :waiting
   229              ct.state = :runnable
   230          end
   231          rethrow()
   232      end
   233  end
   234

test@pts00433-vm1:~/Mahesh/julia/julia$ cat -n ./base/task.jl | head -n 317 | tail -24
   294
   295  function wait()
   296      while true
   297          if isempty(Workqueue)
   298              c = process_events(true)
   299              if c==0 && eventloop()!=C_NULL && isempty(Workqueue)
   300                  # if there are no active handles and no runnable tasks, just
   301                  # wait for signals.
   302                  pause()
   303              end
   304          else
   305              t = shift!(Workqueue)
   306              arg = t.result
   307              t.result = nothing
   308              t.state = :runnable
   309              result = yieldto(t, arg)
   310              process_events(false)
   311              # return when we come out of the queue
   312              return result
   313          end
   314      end
   315      assert(false)
   316  end
   317

Sorry for providing lots of information but I want to give all the information that I have so it will help to understand problem and come to any conclusion/solution.

Thanks & Regards,
-Mahesh

[yuyichao]

Please quote your code blocks.

[yuyichao]

*** Please observe above frame 2 and 3 and which will clearly shows that ./julia binary is in infinite wait.

Why is that? The backtrace looks fine although the wait function (or what happened before this) could have done sth bad.

I guess the wait function never returns but could you please hit finish a few times to make sure?

[vtjnash]

cross link to original julia-users thread: https://groups.google.com/d/msg/julia-users/zYJSlkBiSPE/-qGHNzyLAOIJ

[vtjnash]

this should be fixed now (i forget what the error was, but I've gotten to a REPL on ppc64le on v0.5-dev)