SIGSEGV when running opir #11

planetis-m · 2022-01-04T19:39:37Z

I have installed raylib globally as per https://github.com/raysan5/raylib/wiki/Working-on-GNU-Linux#build-raylib-using-make
My nim code is:

import futhark

# Tell futhark where to find the C libraries you will compile with, and what
# header files you wish to import.
importc:
  absPath "/usr/lib/clang/13.0.0/include/"
  absPath "/usr/local/include"
  "raylib.h"

# Tell Nim how to compile against the library. If you have a dynamic library
# this would simply be a `--passL:"-l<library name>`
{.passL: "-lraylib -lGL -lm -lpthread -ldl -lrt -lX11 -DPLATFORM_DESKTOP".}

# Use the library just like you would in C!
proc main =
  InitWindow(800, 450, "raylib [core] example - basic window")
  while not WindowShouldClose():
    BeginDrawing()
    ClearBackground(RAYWHITE)
    DrawText("Congrats! You created your first window!", 190, 200, 20, LIGHTGRAY)
    EndDrawing()
  CloseWindow()

main()

The output I am getting when running opir -I/usr/local/include -I/usr/lib/clang/13.0.0/include ~/.cache/nim/makeray_r/futhark-includes.h:

Traceback (most recent call last)
futhark/src/opir.nim(343) opir
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
(segmentation fault) (core dumped)

Any way to debug this further?

The text was updated successfully, but these errors were encountered:

PMunch · 2022-01-04T21:52:18Z

Hmm, not entirely sure. Running the exact same code on my machine works fine, albeit erroring out on something else.. Which clang package do you have installed? And I assume you have the right glfw backend installed?

planetis-m · 2022-01-05T13:04:16Z

I didn't have glfw installed, as its bundled with raylib. I have installed my system's glfw (added the /usr/include/GLFW/ in importc) but no change. I package clang 13.0.0-4 from Manjaro installed (stable branch).

PMunch · 2022-01-05T13:06:53Z

I'm also on Manjaro and when I installed raylib it asked me if I wanted to install the glfw dependency as Wayland or X11.

planetis-m · 2022-01-05T13:09:03Z

Till now I was using statically built raylib, I will try again with shared raylib from the Manjaro repo and report back to you.

planetis-m · 2022-01-05T13:14:44Z

I get the same error, however a thing I forgot to report is I don't have nimble installed. I compiled opir and modified futhark to run ./opir in line 458. I just copied opir in my PATH and removed my modification, no difference.

my nim.cfg in futhark directory

############# begin Atlas config section ##########
--noNimblePath
--path:"../futhark/src"
--path:"../libclang-nim/src/"
--path:"../termstyle"
--path:"../macroutils/src"
############# end Atlas config section   ##########

seems to work fine...

planetis-m · 2022-01-05T13:52:47Z

I think the problem lies with libclang-nim, my system has installed libclang.so, but the bindings don't use the dynlib pragma. using passL: "-static -lclang" reveals I don't have a static libclang installed.

PMunch · 2022-01-05T14:11:33Z

Aha, that might explain it, but in that case it's a bit weird that Opir was even able to build. You should be able to just add Opir to your path so you don't have to modify Futhark itself. Shouldn't make a difference though. Which libclang version do you use? I have my own fork where I've fixed some bugs in the official version.

planetis-m · 2022-01-05T14:28:06Z

Which libclang version do you use?

The one provided by manjaro, which is probably a rebuilt of https://archlinux.org/packages/extra/x86_64/clang/

planetis-m · 2022-01-05T14:31:39Z

~~these bindings https://github.com/deech/libclang_bindings seem to provide both a static and a shared version~~

planetis-m · 2022-01-05T18:35:44Z

I added dynlib: "libclang.so" to every function in libclang-nim. still getting the same error...

PMunch · 2022-01-06T08:01:08Z

Here is the Opir output when I run your code on my machine: http://ix.io/3Lcf

PMunch · 2022-01-06T08:02:21Z

By the way, these are my packages:

[peter /tmp ] 16779 $ pacman -Qi clang
Name            : clang
Version         : 13.0.0-4
Description     : C language family frontend for LLVM
Architecture    : x86_64
URL             : https://clang.llvm.org/
Licenses        : custom:Apache 2.0 with LLVM Exception
Groups          : None
Provides        : clang-analyzer=13.0.0  clang-tools-extra=13.0.0
Depends On      : llvm-libs  gcc  compiler-rt
Optional Deps   : openmp: OpenMP support in clang with -fopenmp
                  python: for scan-view and git-clang-format [installed]
                  llvm: referenced by some clang headers [installed]
Required By     : lldb  openshadinglanguage  shiboken2-git
Optional For    : kakoune  qmk  qt5-tools
Conflicts With  : clang-analyzer  clang-tools-extra
Replaces        : clang-analyzer  clang-tools-extra
Installed Size  : 207.42 MiB
Packager        : Felix Yan <[email protected]>
Build Date      : Thu 02 Dec 2021 16:17:59 CET
Install Date    : Tue 04 Jan 2022 09:28:17 CET
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature

[peter /tmp ] 16780 $ pacman -Qi raylib 
Name            : raylib
Version         : 4.0.0-1
Description     : Simple and easy-to-use game programming library
Architecture    : x86_64
URL             : https://www.raylib.com
Licenses        : ZLIB
Groups          : None
Provides        : None
Depends On      : glfw
Optional Deps   : None
Required By     : None
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 1769.90 KiB
Packager        : Alexander Rødseth <[email protected]>
Build Date      : Sun 07 Nov 2021 21:40:15 CET
Install Date    : Tue 04 Jan 2022 22:48:19 CET
Install Reason  : Explicitly installed
Install Script  : No
Validated By    : Signature

planetis-m · 2022-01-06T11:43:18Z

Here is the Opir output when I run your code on my machine: http://ix.io/3Lcf

This is much better than what I was working with thus far, thank you for making futhrak and opir!

PMunch · 2022-01-06T12:25:34Z

Just too bad that Opir crashes on your machine. These kinds of errors are also super hard for me to debug since I can't reproduce the error..

arkanoid87 · 2022-01-12T06:58:00Z

Futhark was working nice with my project, then I accepted to install ubuntu upgrades on my 20.04 LTS and suddenly I face same SIGSEGV: Illegal storage access. (Attempt to read from nil?) on opir execution

The change should be outside futhark and libclang-nim

Thanks to a stable setup before the upgrade, I recompiled opir in debug mode traced down the problem to the line:

nimble install futhark@#head

project:
https://github.com/arkanoid87/nimmap

cmdline:
opir -I/usr/lib/clang/10/include -I/usr/include/gdal /home/jack/.cache/nim/nimmap_d/futhark-includes.h

futhark-includes.h:

#include "gdal.h"
#include "ogr_core.h"
#include "ogr_api.h"

nim output:

Traceback (most recent call last)
/home/jack/.nimble/pkgs/futhark-#head/opir.nim(343) opir
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

futhark/src/opir.nim

Line 343 in 875a7f5

var cursor = getTranslationUnitCursor(unit)

(gdb) run -I/usr/lib/clang/10/include -I/usr/include/gdal /home/jack/.cache/nim/nimmap_d/futhark-includes.h

Starting program: /home/jack/.nimble/pkgs/futhark-#head/opir -I/usr/lib/clang/10/include -I/usr/include/gdal /home/jack/.cache/nim/nimmap_d/futhark-includes.h
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff119f700 (LWP 15886)]
[Thread 0x7ffff119f700 (LWP 15886) exited]

Thread 1 "opir" received signal SIGSEGV, Segmentation fault.
0x00007ffff62a81d3 in clang_getTranslationUnitCursor () from /usr/lib/x86_64-linux-gnu/libclang-10.so.1
(gdb) bt
#0  0x00007ffff62a81d3 in clang_getTranslationUnitCursor () from /usr/lib/x86_64-linux-gnu/libclang-10.so.1
#1  0x0000555555592c52 in NimMainModule ()
#2  0x000055555559197e in NimMainInner ()
#3  0x00005555555919be in NimMain ()
#4  0x0000555555591a10 in main ()

seems to happen inside clang_getTranslationUnitCursor

dump repr unit (of var cursor = getTranslationUnitCursor(unit))

repr unit = 0x7f17480036d0

stat /usr/lib/x86_64-linux-gnu/libclang-10.so.1

  File: /usr/lib/x86_64-linux-gnu/libclang-10.so.1
  Size: 34221680        Blocks: 66840      IO Block: 4096   regular file
Device: 815h/2069d      Inode: 8392614     Links: 2
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2022-01-12 06:09:06.795501871 +0100
Modify: 2020-04-20 07:12:09.000000000 +0200
Change: 2021-12-06 03:28:40.168317608 +0100

nm -D /usr/lib/x86_64-linux-gnu/libclang-10.so.1 | grep clang_getTranslationUnitCursor

00000000003cf1c0 T clang_getTranslationUnitCursor

debugging var unit

...
import sugar
dump repr index
dump fname
for i in 0..args:
  dump commandLine[i]
dump args
var unit = parseTranslationUnit(index, fname.cstring,
                              commandLine, args.cint, nil, 0, CXTranslationUnit_DetailedPreprocessingRecord.cuint or CXTranslationUnit_SkipFunctionBodies.cuint)
dump unit.getNumDiagnostics
..

repr index = 0x5653344db180
fname = /tmp/imports-20501.h
commandLine[i] = -I/usr/lib/clang/10/include
commandLine[i] = -I/usr/include/gdal
commandLine[i] = /home/jack/.cache/nim/nimmap_d/futhark-includes.h
args = 2
unit.getNumDiagnostics = 0

cat /tmp/imports-20501.h

#include "/home/jack/.cache/nim/nimmap_d/futhark-includes.h"

not quite sure what's going on, seems that the output of clang_parseTranslationUnit is correct and goes straight into clang_getTranslationUnitCursor tha goes SIGSEGV

PMunch · 2022-01-12T07:35:27Z

Yeah, I also got that far when I was debugging this. That's why I was trying to find if we had any differences in packages or setup. Not really sure what to do about this.. It might be that the Clang wrapper is incorrect?

arkanoid87 · 2022-01-12T07:44:59Z

just tried compiling and linking opir against libclang-12 instead of libclang-10, but I got same results

I was actually trying to figure out what happened 2 hours ago when I installed ubuntu upgrades via gnome apt, but with my surprise there's no info of that operation in /var/log/apt

arkanoid87 · 2022-01-12T21:24:01Z

In my case, the same problem happens on both libclang-10, 11 and 12, so I'd stick with 10 as I've always use it for the past months on different futhark based projects.

I've installed the debug symbols, so I've a line to blame:
#0 clang_getTranslationUnitCursor () at /build/llvm-toolchain-10-yegZYJ/llvm-toolchain-10-10.0.0/clang/tools/libclang/CIndex.cpp:4208

I've grabbed the original file source version

CXCursor clang_getTranslationUnitCursor(CXTranslationUnit TU) {
  if (isNotUsableTU(TU)) {
    LOG_BAD_TU(TU);
    return clang_getNullCursor();
  }

  ASTUnit *CXXUnit = cxtu::getASTUnit(TU);
  return MakeCXCursor(CXXUnit->getASTContext().getTranslationUnitDecl(), TU); // 4208
}

edit: found same ref line on github https://github.com/llvm/llvm-project/blob/ef32c611aa214dea855364efd7ba451ec5ec3f74/clang/tools/libclang/CIndex.cpp#L4208

considering the fact that gdb bt shows that SIGSEGV happens in clang_getTranslationUnitCursor, it means that isNotUsableTU(TU) returns false but CXXUnit end up invalid pointer, so the issue might be cxtu::getASTUnit

arkanoid87 · 2022-01-12T22:31:40Z

diving into the issue

edit in index.nim (cland-#head pkg) to reflect original struct

type
  # CXTranslationUnit* = pointer # CXTranslationUnitImpl
  CXTranslationUnit* = ptr object # CXTranslationUnitImpl
    cIdx*: CXIndex
    theASTUnit*: pointer
    cXStringPool*: pointer
    diagnostic*: pointer
    overridenCursorsPool*: pointer
    commentToXML*: pointer
    parsingOptions*: cuint
    arguments: pointer

checks in opir.nim

var unit = parseTranslationUnit(index, fname.cstring,
                              commandLine, args.cint, nil, 0, CXTranslationUnit_DetailedPreprocessingRecord.cuint or CXTranslationUnit_SkipFunctionBodies.cuint)
deallocCStringArray(commandLine)

assert not unit.isNil
# assert first and last struct fields
assert unit.cIdx == index
assert unit.parsingOptions == (CXTranslationUnit_DetailedPreprocessingRecord.cuint or CXTranslationUnit_SkipFunctionBodies.cuint)
#
assert not unit.theASTUnit.isNil
echo repr unit.theASTUnit

output

nim c -r futhest.nim
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.2/config/nim.cfg' [Conf]
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.2/config/config.nims' [Conf]
........................................................................................................
/home/jack/.nimble/pkgs/futhark-#head/futhark.nim(479, 12) Hint: Running: opir -I/usr/lib/clang/10/include -I/home/jack/nim/futhest/stb /home/jack/.cache/nim/futhest_d/futhark-includes.h [User]
0x7f06d0002b00
Traceback (most recent call last)
/home/jack/.nimble/pkgs/futhark-#head/opir.nim(351) opir
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
/home/jack/.nimble/pkgs/futhark-#head/futhark.nim(486, 8) Hint: Parsing Opir output [User]
stack trace: (most recent call last)
/home/jack/.nimble/pkgs/futhark-#head/futhark.nim(494, 11) importcImpl
/home/jack/nim/futhest/futhest.nim(5, 1) template/generic instantiation of `importc` from here
/home/jack/.nimble/pkgs/futhark-#head/futhark.nim(440, 14) template/generic instantiation of `importcImpl` from here
/home/jack/.nimble/pkgs/futhark-#head/futhark.nim(494, 11) Error: Unable to parse output of opir:
Segmentation fault (core dumped)

at least we know that unit.theASTUnit is not nil

arkanoid87 · 2022-01-12T23:19:24Z

created minimal example to reproduce the error:

import clang


static:
  # any content will fail, including empty file
  writeFile("futhark-includes.h", """
  #include "gdal.h"
""")

var
  commandLineParams = @[
    "-I/usr/lib/clang/10/include",
    "-I/usr/include/gdal"]
  fname = "futhark-includes.h"

  index = createIndex(0, 0)
  commandLine = allocCStringArray(commandLineParams)


var unit = parseTranslationUnit(index, fname.cstring,
                              commandLine, commandLineParams.len.cint, nil, 0, CXTranslationUnit_DetailedPreprocessingRecord.cuint or CXTranslationUnit_SkipFunctionBodies.cuint)
deallocCStringArray(commandLine)

block: # testing stuff
    assert not unit.isNil
    assert unit.getNumDiagnostics == 0
    # assert first and last struct fields
    assert unit.cIdx == index
    assert unit.parsingOptions == (CXTranslationUnit_DetailedPreprocessingRecord.cuint or CXTranslationUnit_SkipFunctionBodies.cuint)

    assert not unit.theASTUnit.isNil
    echo repr unit.theASTUnit
    echo unit.getTranslationUnitSpelling
    echo repr unit.getCXTUResourceUsage

discard getTranslationUnitCursor(unit) # SIGSEGV
assert false

output

nim c -r futhest.nim
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.2/config/nim.cfg' [Conf]
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.2/config/config.nims' [Conf]
Hint: used config file '/home/jack/nim/futhest/config.nims' [Conf]
Hint: gc: refc; opt: none (DEBUG BUILD, `-d:release` generates faster code)
19404 lines; 0.019s; 16.559MiB peakmem; proj: /home/jack/nim/futhest/futhest.nim; out: /home/jack/nim/futhest/futhest [SuccessX]
Hint: /home/jack/nim/futhest/futhest  [Exec]
0x7fbfcc002b00
(data: ..., private_flags: 1)
[data = 0x55c328a6d700,
numEntries = 12,
entries = ptr 0x55c328a58120 --> [kind = CXTUResourceUsage_AST,
amount = 4947968]]
Traceback (most recent call last)
/home/jack/nim/futhest/futhest.nim(66) futhest
SIGSEGV: Illegal storage access. (Attempt to read from nil?)
Segmentation fault (core dumped)
Error: execution of an external program failed: '/home/jack/nim/futhest/futhest '

arkanoid87 · 2022-01-13T00:09:06Z

I've created a docker based nimble package to replicate the issue and exclude single-machine case

https://github.com/arkanoid87/futhest

it includes 2 Dockerfile to replicate error on ubuntu and alpine-linux

I've also just included a working plain C example that does not crash

#include <assert.h>
#include <stdio.h>
#include "clang-c/Index.h"


enum CXChildVisitResult
visitor(CXCursor cursor, CXCursor parent, CXClientData clientData) {
    CXFile file;
    unsigned int line;
    unsigned int column;
    unsigned int offset;

    CXSourceLocation loc = clang_getCursorLocation(cursor);
    clang_getFileLocation(loc, &file, &line, &column, &offset);

    CXString filename = clang_getFileName(file);   
    printf("%s [%d:%d, %d]\n", clang_getCString(filename), line, column, offset);
    clang_disposeString(filename);

    return CXChildVisit_Continue;
}

int main (void) { 
    CXIndex Idx = clang_createIndex(0, 0);

    CXTranslationUnit TU = clang_parseTranslationUnit(Idx, 
        "src/header.h", NULL, 0, NULL, 0, 0);

    assert(TU != NULL);

    CXCursor cursor = clang_getTranslationUnitCursor(TU);

    clang_visitChildren(cursor, visitor, NULL);

    clang_disposeTranslationUnit(TU);

    printf("END");
}

arkanoid87 · 2022-01-13T04:09:09Z

Relevant forum post: https://forum.nim-lang.org/t/8796

EDIT: solution found, is a regression in nim compiler

1.7.1 devel 9888a29c3de09c73202b2a955e09e2a7b5e56ea4: SIGSEGV
1.6.2: SIGSEGV
1.6.0: OK
1.4.8: OK
1.2.0: OK
1.0.0: OK

remember to clean your ~/.cache/nim/project_* folder after swtiching

New issue: nim-lang/Nim#19378

PMunch · 2022-01-13T07:16:02Z

Aha! My machine was still running Nim 1.6.0 which was why I didn't see the error. Hopefully this will be fixed quickly!

PMunch · 2022-01-13T13:10:49Z

Might be, NimLSP (which is what I'm using for Vim) embeds the Nim compiler and calls it directly in the same way that nimsuggest does.

ringabout · 2022-01-13T14:52:10Z

related issue
nim-lang/Nim#19342

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SIGSEGV when running opir #11

SIGSEGV when running opir #11

planetis-m commented Jan 4, 2022 •

edited

Loading

PMunch commented Jan 4, 2022

planetis-m commented Jan 5, 2022 •

edited

Loading

PMunch commented Jan 5, 2022

planetis-m commented Jan 5, 2022

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

PMunch commented Jan 5, 2022

planetis-m commented Jan 5, 2022

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

PMunch commented Jan 6, 2022

PMunch commented Jan 6, 2022

planetis-m commented Jan 6, 2022 •

edited

Loading

PMunch commented Jan 6, 2022

arkanoid87 commented Jan 12, 2022

PMunch commented Jan 12, 2022

arkanoid87 commented Jan 12, 2022

arkanoid87 commented Jan 12, 2022 •

edited

Loading

arkanoid87 commented Jan 12, 2022

arkanoid87 commented Jan 12, 2022

arkanoid87 commented Jan 13, 2022 •

edited

Loading

arkanoid87 commented Jan 13, 2022 •

edited

Loading

PMunch commented Jan 13, 2022

This comment has been minimized.

PMunch commented Jan 13, 2022

ringabout commented Jan 13, 2022 •

edited

Loading

ringabout commented Jan 21, 2022 •

edited

Loading

arkanoid87 commented Jan 21, 2022

PhilippMDoerner commented Feb 6, 2022

arkanoid87 commented Feb 6, 2022

PMunch commented Mar 3, 2022

SIGSEGV when running opir #11

SIGSEGV when running opir #11

Comments

planetis-m commented Jan 4, 2022 • edited Loading

PMunch commented Jan 4, 2022

planetis-m commented Jan 5, 2022 • edited Loading

PMunch commented Jan 5, 2022

planetis-m commented Jan 5, 2022

planetis-m commented Jan 5, 2022 • edited Loading

planetis-m commented Jan 5, 2022 • edited Loading

PMunch commented Jan 5, 2022

planetis-m commented Jan 5, 2022

planetis-m commented Jan 5, 2022 • edited Loading

planetis-m commented Jan 5, 2022 • edited Loading

PMunch commented Jan 6, 2022

PMunch commented Jan 6, 2022

planetis-m commented Jan 6, 2022 • edited Loading

PMunch commented Jan 6, 2022

arkanoid87 commented Jan 12, 2022

PMunch commented Jan 12, 2022

arkanoid87 commented Jan 12, 2022

arkanoid87 commented Jan 12, 2022 • edited Loading

arkanoid87 commented Jan 12, 2022

arkanoid87 commented Jan 12, 2022

arkanoid87 commented Jan 13, 2022 • edited Loading

arkanoid87 commented Jan 13, 2022 • edited Loading

PMunch commented Jan 13, 2022

This comment has been minimized.

PMunch commented Jan 13, 2022

ringabout commented Jan 13, 2022 • edited Loading

ringabout commented Jan 21, 2022 • edited Loading

arkanoid87 commented Jan 21, 2022

PhilippMDoerner commented Feb 6, 2022

arkanoid87 commented Feb 6, 2022

PMunch commented Mar 3, 2022

planetis-m commented Jan 4, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 5, 2022 •

edited

Loading

planetis-m commented Jan 6, 2022 •

edited

Loading

arkanoid87 commented Jan 12, 2022 •

edited

Loading

arkanoid87 commented Jan 13, 2022 •

edited

Loading

arkanoid87 commented Jan 13, 2022 •

edited

Loading

ringabout commented Jan 13, 2022 •

edited

Loading

ringabout commented Jan 21, 2022 •

edited

Loading