trouble wraping glib

[bjarthur]

the master branch (fa22244) of clang is choking on a #define with special characters in it:

#define G_URI_RESERVED_CHARS_SUBCOMPONENT_DELIMITERS "!$&'()*+,;="

the error is:

[ Info: Error thrown. Last cursor available in Clang.wrap_c.debug_cursors
ERROR: Base.Meta.ParseError("invalid interpolation syntax: \"\$&\"")
Stacktrace:
 [1] #parse#1(::Bool, ::Bool, ::Bool, ::Function, ::String, ::Int64) at ./meta.jl:129
 [2] #parse at ./none:0 [inlined]
 [3] #parse#4(::Bool, ::Bool, ::Function, ::String) at ./meta.jl:164
 [4] parse at ./meta.jl:164 [inlined]
 [5] wrap(::Clang.wrap_c.WrapContext, ::OrderedCollections.OrderedDict{Symbol,Clang.wrap_c.ExprUnit}, ::Clang.cindex.MacroDefinition) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:630
 [6] wrap_header(::Clang.wrap_c.WrapContext, ::Clang.cindex.TranslationUnit, ::String, ::Array{Any,1}) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:684
 [7] run(::Clang.wrap_c.WrapContext) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:801
 [8] top-level scope at none:0

to reproduce, first install Cairo.jl, which is how i got the glib header files, and how i'm hoping to wrap all of libpangocairo, and then:

using Clang

context = wrap_c.init(
  clang_includes=[
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/pango/1.42.4/include/pango-1.0",
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/glib/2.58.0_1/include/glib-2.0",
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/glib/2.58.0_1/lib/glib-2.0/include",
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/cairo/1.14.12/include/cairo"],
  headers=[
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/pango/1.42.4/include/pango-1.0/pango/pangocairo.h"],
  common_file="pangocairo-constants.jl",
  header_library=x->"_jl_libpangocairo")

run(context)

it's worth nothing that simply using Clang results in the following warning:

WARNING: could not import Base.search into cindex

any ideas? i'd be happy with a work around. thanks.

[bjarthur]

forgot to mention that the offending line is number 44 of

/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellarglib/2.58.0_1/include/glib-2.0/glib/gurifuncs.h.

i infer this for if i change line 630 of Clang/src/wrap_c.jl to be:

local target
try
    target = Meta.parse(exprn)
catch
@info exprn
end

and then run the reproduction example above again, i get:

[ Info: "!$&'()*+,;="
[ Info: Error thrown. Last cursor available in Clang.wrap_c.debug_cursors
ERROR: UndefVarError: target not defined
Stacktrace:
 [1] macro expansion at ./logging.jl:307 [inlined]
 [2] wrap(::Clang.wrap_c.WrapContext, ::OrderedCollections.OrderedDict{Symbol,Clang.wrap_c.ExprUnit}, ::Clang.cindex.MacroDefinition) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:634
 [3] wrap_header(::Clang.wrap_c.WrapContext, ::Clang.cindex.TranslationUnit, ::String, ::Array{Any,1}) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:689
 [4] run(::Clang.wrap_c.WrapContext) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:806
 [5] top-level scope at none:0

searching the header files yields only one line with those special characters:

$ grep -r '!$&' glib/2.58.0_1/include/glib-2.0
glib/2.58.0_1/include/glib-2.0/glib/gurifuncs.h: * Subcomponent delimiter characters as defined in RFC 3986. Includes "!$&'()*+,;=".
glib/2.58.0_1/include/glib-2.0/glib/gurifuncs.h:#define G_URI_RESERVED_CHARS_SUBCOMPONENT_DELIMITERS "!$&'()*+,;="
glib/2.58.0_1/include/glib-2.0/glib/gurifuncs.h: * Allowed characters in path elements. Includes "!$&'()*+,;=:@".
glib/2.58.0_1/include/glib-2.0/glib/gurifuncs.h: * Allowed characters in a path. Includes "!$&'()*+,;=:@/".
glib/2.58.0_1/include/glib-2.0/glib/gurifuncs.h: * Allowed characters in userinfo as defined in RFC 3986. Includes "!$&'()*+,;=:".

$ grep -r '!$&' pango/1.42.4/include/pango-1.0/pango

$ grep -r '!$&' cairo/1.14.12/include/cairo

[Gnimuc]

that's because "!$&'()*+,;=" is not a valid string literal in Julia. A possible fix would be:

try
    target = Meta.parse(exprn)
    e = Expr(:const, Expr(:(=), use_sym, target))
catch err
    e = :(const $use_sym = $(exprn[2:end-1]))
end

[bjarthur]

thanks for the quick reply! Clang gets much further with a slight modification to your code snippet. i had to alter it because target gets used a couple lines later as input to ExprUnit.

try
    target = Meta.parse(exprn)
    e = Expr(:const, Expr(:(=), use_sym, target))
    deps = get_symbols(target)
    expr_buf[use_sym] = ExprUnit(e, deps)
catch err
    e = :(const $use_sym = $(exprn[2:end-1]))     # or :(const $use_sym = raw$(exprn))  ?
    expr_buf[use_sym] = ExprUnit()                # or ExprUnit(e,[])   ?
end 

now it's getting stuck with the following trace. same trace is emitted when i use raw$(exprn) or ExprUnit(e,[]).

[ Info: Error thrown. Last cursor available in Clang.wrap_c.debug_cursors
ERROR: LoadError: BoundsError: attempt to access 0-element Array{Clang.cindex.CLCursor,1} at index [0]
Stacktrace:
 [1] getindex at ./array.jl:731 [inlined]
 [2] largestfield(::Clang.cindex.UnionDecl) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:311
 [3] #wrap#32(::String, ::Function, ::Clang.wrap_c.WrapContext, ::OrderedCollections.OrderedDict{Symbol,Clang.wrap_c.ExprUnit}, ::Clang.cindex.UnionDecl) at ./logging.jl:310
 [4] wrap(::Clang.wrap_c.WrapContext, ::OrderedCollections.OrderedDict{Symbol,Clang.wrap_c.ExprUnit}, ::Clang.cindex.UnionDecl) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:435
 [5] wrap_header(::Clang.wrap_c.WrapContext, ::Clang.cindex.TranslationUnit, ::String, ::Array{Any,1}) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:710
 [6] run(::Clang.wrap_c.WrapContext) at /Users/arthurb/.julia/dev/Clang/src/wrap_c.jl:827
 [7] top-level scope at none:0
 [8] include at ./boot.jl:317 [inlined]
 [9] include_relative(::Module, ::String) at ./loading.jl:1038
 [10] include(::Module, ::String) at ./sysimg.jl:29
 [11] include(::String) at ./client.jl:388
 [12] top-level scope at none:0

[Gnimuc]

that looks like an orthogonal error, did you find which line in the header file that triggered this error?

[Gnimuc]

I just found this issue is actually a dup of #196 (comment). It looks like you were wrapping libpangocairo and its dependencies(e.g. glib). If you just want libpangocairo, try the following script instead:

using Clang

context = wrap_c.init(
  clang_includes=[
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/pango/1.42.4/include/pango-1.0",
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/glib/2.58.0_1/include/glib-2.0",
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/glib/2.58.0_1/lib/glib-2.0/include",
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/cairo/1.14.12/include/cairo"],
  headers=[
    "/Users/arthurb/.julia/packages/Homebrew/l8kUw/deps/usr/Cellar/pango/1.42.4/include/pango-1.0/pango/pangocairo.h"],
  common_file="pangocairo-constants.jl",
  header_library = x->"_jl_libpangocairo",
  header_wrapped = (top_header, cursor_header) -> top_header == cursor_header)
)

run(context)

[bjarthur]

thanks! making progress. now the problem is that Clang is outputing const clock_t = __darwin_clock_t and of course that is not defined. i find it strange that there is no Cclock_t defined in julia to match Csize_t. any suggestions on what bitstype to use that should be robust across systems?

[Gnimuc]

those compiler definitions are skipped:

https://github.com/ihnorton/Clang.jl/blob/fa22244d21a18e14ec301bd410d88140c280de40/src/wrap_c.jl#L674

Currently, Clang.jl is mainly for trivial jobs, it's common to manually tweak the output code.

[Gnimuc]

hi @bjarthur, Clang.jl master now ships a ctypes.jl file which includes clock_t definition.