Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build fails on OpenBSD #965

Closed
lkppo opened this issue Mar 20, 2019 · 27 comments
Closed

Build fails on OpenBSD #965

lkppo opened this issue Mar 20, 2019 · 27 comments
Assignees

Comments

@lkppo
Copy link
Collaborator

lkppo commented Mar 20, 2019

A build using a binary compiled with commit 4a01d0e fails with the last commit on:

$ ./r3-openbsd-x64 make.r CONFIG=configs/generic.r OS-ID=0.9.40
[
[M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS]
[M32 ARC PIE HID DYN CON S4M]
[LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER]
[M DL LOG W32 NWK]]
** Error: Unused flags in %systems.r specifications
** Where: fail if use do catch either else _ do do catch either else _ do console
** Near: [... unused-flags
fail "Unused flags in %systems.r speci...
** File: tools/systems.r
** Line: 495

I change the code arround unused-flags in %tools/systems.r like that:

unused-flags: exclude compose [
    ((words-of compiler-flags))
    ((words-of linker-flags))
    ((words-of system-definitions))
    ((words-of system-libraries))
] used-flags

print "compiler-flags"
print mold (words-of compiler-flags)

print "linker-flags"
print mold (words-of linker-flags)

print "system-definitions"
print mold (words-of system-definitions)

print "system-libraries"
print mold (words-of system-libraries)

print "used-flag"
print mold used-flags

print "unused-flag"
print mold unused-flags

 print "test compose"
 print mold compose [
    ((words-of compiler-flags))
    ((words-of linker-flags))
    ((words-of system-definitions))
    ((words-of system-libraries))
]

if not empty? unused-flags [
    print mold unused-flags
    fail "Unused flags in %systems.r specifications"
]

On OpenBSD the build fails and the modified output is:

vps642026$ ./r3-openbsd-x64 make.r CONFIG=configs/generic.r
compiler-flags
[M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS]
linker-flags
[M32 ARC PIE HID DYN CON S4M]
system-definitions
[LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER]
system-libraries
[M DL LOG W32 NWK]
used-flag
[
SGD BEN LLC F64 M NPS HID DYN NCM LEN NSER ARC UNI W32 NSEC WLOSS CON S4M LLP64 DL M32 NSP UFS PIP2 LP64 PIE NWK LOG PIC
SG?]
unused-flag
[
[M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS]
[M32 ARC PIE HID DYN CON S4M]
[LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER]
[M DL LOG W32 NWK]]
test compose
[
[M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS]
[M32 ARC PIE HID DYN CON S4M]
[LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER]
[M DL LOG W32 NWK]
]
[
[M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS]
[M32 ARC PIE HID DYN CON S4M]
[LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER]
[M DL LOG W32 NWK]]
** Error: Unused flags in %systems.r specifications
** Where: fail if use do catch either else _ do do catch either else _ do console
** Near: [... unused-flags
fail "Unused flags in %systems.r speci...
** File: tools/systems.r
** Line: 495

On Windows the buid success and the modified output is:

$ ./r3-windows-x86-8994d23 make.r CONFIG=configs/generic.r
== SHIMMING OLDER R3 TO MODERN LANGUAGE DEFINITIONS ==
compiler-flags
[M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS]
linker-flags
[M32 ARC PIE HID DYN CON S4M NOWASM]
system-definitions
[LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER]
system-libraries
[M DL LOG W32 NWK]
used-flag
[
SGD BEN LLC F64 M NPS HID DYN NCM LEN NSER ARC UNI W32 NSEC WLOSS CON S4M LLP64 DL M32 NSP UFS PIP2 LP64 PIE NWK LOG PIC
SG? NOWASM]
unused-flag
[]
test compose
[
M32 ARC HID NPS PIE NCM UFS NSP PIC WLOSS
M32 ARC PIE HID DYN CON S4M NOWASM
LP64 LLP64 BEN LEN LLC SGD SG? W32 UNI F64 NSEC PIP2 NSER
M DL LOG W32 NWK
]
make object! [

I think commit is 31aa9be#diff-2af3fa21d38da56bb8293d98b0453b85 changed the behavior of COMPOSE but I don't know why windows can compile with r3-windows-x86-8994d23.exe which older that my binary.

If I bypass unused-flag test the build fails just after this point:

Sanity checking on system config....Good
Sanity checking on app config....Good
definitions: ["NDEBUG"
"OS_STACK_GROWS_DOWN" "ENDIAN_LITTLE" "HAS_LL_CONSTS" "_FILE_OFFSET_BITS=64" "LP64"]
includes: [%../src/include %prep/include]
libraries: [make object! [
class: #dynamic-extension
output: gnu:m
flags: _
]]
cflags: []
ldflags: []
debug: false
optimization: 2
** Access Error: value is CONST (see MUTABLE): ["REB_API"]
** Where: repend case join construct if _ make do catch either else _ do console
** Near: [repend series :value ~~]
** File: tmp-boot.r
** Line: 4428

Did I have to attempt a build with the old commit 8994d23 before trying to build last commit?

@hostilefork hostilefork self-assigned this Mar 20, 2019
@hostilefork
Copy link
Member

hostilefork commented Mar 20, 2019

Did I have to attempt a build with the old commit 8994d23 before trying to build last commit?

In general, yes, that is the case.

There are only two versions that should be "guaranteed" to build--the one snapshotted, and the current.

Because those two versions both work, it's usually the case that an intermediate version can--with some effort--be tweaked around to work if absolutely necessary. But it may well not work as-is.

So try building with the older version, and then seeing if the build product of that process can bootstrap.

This does mean that if you want a stable OpenBSD build you might have to go back and build that older version, otherwise you'll have to periodically re-bootstrap.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 20, 2019

Ok, I will give a try this previous commit. If it works I have to add it to the repository.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

I bootstrapped again successfully an pushed the binary in pull request #968

Now this binary can build with the last commit but its product segfault immediatly.

I have only a partial log.

@metaeducation metaeducation deleted a comment from lkppo Mar 22, 2019
@hostilefork
Copy link
Member

hostilefork commented Mar 22, 2019

I have only a partial log.

Thanks, I saved it as a file. Is there any way you can run with the debug=sanitize option? Address Sanitizer gives some decent dump of the stack when it runs, could give a hint.

(If you're handy with a debugger of any sort enough to get a stack of any kind, even a screenshot of one, it could offer a clue)

@hostilefork
Copy link
Member

Actually, I note that's not a debug build. At least do debug=asserts, it's pretty noisy about when things are going wrong.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

I am trying to intercept the output in a file but the redirection of the standard output does not work with the parameters. I don't use the last Firefox and github block the file upload, sorry.

I tested a lot of parameters. Do you want "make.r CONFIG=configs/generic.r DEBUG=asserts" or something else?

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

vps642026$ time ./r3-openbsd-x64-8994d23 make.r CONFIG=configs/generic-c++.r DEBUG=sanitize

Causes a bunch of stuff, but none of the compile steps succeed because of this error:

cc1plus: error: unrecognized command line option "-fsanitize=address"

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

vps642026$ gdb r3 r3.core
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-unknown-openbsd6.4"...(no debugging symbols found)

Core was generated by `r3'.
Program terminated with signal 11, Segmentation fault.
(no debugging symbols found)
Loaded symbols for /home/lkppo/ren-c/make/r3
Reading symbols from /usr/lib/libm.so.10.1...done.
Loaded symbols for /usr/lib/libm.so.10.1
Reading symbols from /usr/lib/libc.so.92.5...done.
Loaded symbols for /usr/lib/libc.so.92.5
Reading symbols from /usr/libexec/ld.so...done.
Loaded symbols for /usr/libexec/ld.so
#0 0x00000efbff5ad8a8 in Scan_Quote_Push_Mold () from /home/lkppo/ren-c/make/r3

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

(gdb) bt full
#0 0x00000efbff5ad8a8 in Scan_Quote_Push_Mold () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#1 0x00000efbff5b1ff3 in Locate_Token_May_Push_Mold () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebolsource#2 0x00000efbff5b2e43 in Scan_To_Stack () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebolsource#3 0x00000efbff5b8e8b in Scan_Child_Array () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#4 0x00000efbff5b31df in Scan_To_Stack () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#5 0x00000efbff5b8e8b in Scan_Child_Array () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebolsource#6 0x00000efbff5b31df in Scan_To_Stack () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebol#7 0x00000efbff5b7a9d in Scan_UTF8_Managed () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#8 0x00000efbff692f2e in MAKE_Array () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#9 0x00000efbff5ba26e in N_make () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#10 0x00000efbff53d325 in Eval_Internal_Maybe_Stale_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#11 0x00000efbff537dfe in Rightward_Evaluate_Nonvoid_Into_Out_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#12 0x00000efbff53c269 in Eval_Internal_Maybe_Stale_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#13 0x00000efbff5da986 in Do_Any_Array_At_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#14 0x00000efbff5e42c0 in Do_Branch_Core_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#15 0x00000efbff5e6b9f in N_if () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#16 0x00000efbff53d325 in Eval_Internal_Maybe_Stale_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebol#17 0x00000efbff5da986 in Do_Any_Array_At_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#18 0x00000efbff5e42c0 in Do_Branch_Core_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebol#19 0x00000efbff5e6b9f in N_if () from /home/lkppo/ren-c/make/r3
No symbol table info available.
#20 0x00000efbff53d325 in Eval_Internal_Maybe_Stale_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebol#21 0x00000efbff54cbfa in Do_Any_Array_At_Throws () from /home/lkppo/ren-c/make/r3
No symbol table info available.
rebol#22 0x00000efbff54db49 in Returner_Dispatcher () from /home/lkppo/ren-c/make/r3

@hostilefork
Copy link
Member

hostilefork commented Mar 22, 2019

Thanks...it's certainly more informative than just a crash.

If you saw my digest of the log above, I guess OpenBSD doesn't support address sanitizer (at least not out of the box). You might need a newer GCC or need to do some stuff to get it. But really it wouldn't do too much better than this, so don't worry about it.

I said "at least debug=asserts... there's another step above that which is debug=normal. Which would give you a full debugger, in terms of having symbols.

But with something like this, I should probably just debug it...as you've already done the bootstrap. If it's not too hard to install OpenBSD in VirtualBox, where should I get an image from to be compatible with you?

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

On OpenBSD GCC is customized. A pristine GCC can be installed.

I rebuild with DEBUG=normal

I use the 6.4 ISO in https://cdn.openbsd.org/pub/OpenBSD/6.4/amd64/

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

No, I think the default compiler is LLVM with a masquarade of command to simulate GCC.

Never used a debugger before.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

With DEBUG=normal

vps642026$ gdb r3 r3.core
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "amd64-unknown-openbsd6.4"...
Core was generated by `r3'.
Program terminated with signal 11, Segmentation fault.
Loaded symbols for /home/lkppo/ren-c/make/r3
Reading symbols from /usr/lib/libm.so.10.1...done.
Loaded symbols for /usr/lib/libm.so.10.1
Reading symbols from /usr/lib/libc.so.92.5...done.
Loaded symbols for /usr/lib/libc.so.92.5
Reading symbols from /usr/libexec/ld.so...done.
Loaded symbols for /usr/libexec/ld.so
#0 0x00000fb53e8ad8a8 in Scan_Quote_Push_Mold (mo=0x7f7ffffbd530,
src=0xfb7b9c3e174 "▒\205land Islands"\n "AL" "Albania"\n "DZ" "Algeria"\n "AS" "American Samoa"\n "AD" "Andorra"\n "AO" "Angola"\n "AI" "Anguilla"\n
"AQ" "Antarctica"\n "AG" "Antigua And Barbuda"\n "AR" ""..., ss=0x7f7ffffbd5c0) at sys-char.h:202
202 REBCNT trail = trailingBytesForUTF8[*source];

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

(gdb) bt full
#0 0x00000fb53e8ad8a8 in Scan_Quote_Push_Mold (mo=0x7f7ffffbd530,
src=0xfb7b9c3e174 "▒\205land Islands"\n "AL" "Albania"\n "DZ" "Algeria"\n "AS" "American Samoa"\n "AD" "Andorra"\n "AO" "Angola"\n "AI" "Anguilla"\n
"AQ" "Antarctica"\n "AG" "Antigua And Barbuda"\n "AR" ""..., ss=0x7f7ffffbd5c0) at sys-char.h:202
c = 0
term = 34
nest = 0
lines = 0
func = "Scan_Quote_Push_Mold"
#1 0x00000fb53e8b1ff3 in Locate_Token_May_Push_Mold (mo=0x7f7ffffbd530, ss=0x7f7ffffbd5c0) at ../src/core/l-scan.c:1017
p = Variable "p" is not available.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

(gdb) l
1017 cp = Scan_Quote_Push_Mold(mo, cp, ss);
1018
1019 check_str:
1020 if (cp) {
1021 ss->end = cp;
1022 return TOKEN_STRING;
1023 }
1024 // try to recover at next new line...
1025 cp = ss->begin + 1;
1026 while (not ANY_CR_LF_END(*cp))

@hostilefork
Copy link
Member

I use the 6.4 ISO in https://cdn.openbsd.org/pub/OpenBSD/6.4/amd64/

So you're intentionally running gcc 4.2.1, from 2007? :-/

While I wait for this to compile and poke around with it...perhaps you could post an introduction and explain your motivations a bit...so I understand why I'm doing it!

https://forum.rebol.info/t/introductions-who-am-i/197

@hostilefork
Copy link
Member

I'm not having any trouble (well, unless you count not being able to copy or paste out of the terminal as trouble, I don't have a "middle mouse button").

What I did was to install the install64.iso from the location you gave, then I did pkg_add git. I cloned your repo and switched to the branch with the OpenBSD build.

With that old a compiler, C++ serves no point. You need C++11. (There were compilers around that would work prior to 2011, which would say --std=c++0x, but this is too old even for that.) So don't bother trying to build as C++ with this compiler.

You might try deleting everything and re-enlisting in your branch. The simple command:

 ./r3-openbsd-x64-8994d23 make.r

Worked fine for me. But I tried various optimization levels, debug and not, generic config/etc. The things that don't work are sanitizing and C++11. If you steer clear of that it should work.

So try it from a clean clone of your branch with the bootstrap exe, check your gcc --version and say something if it's not 4.2.1. Then just try a simple make.r call like the above.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 22, 2019

Cross answered with ticket #960.

That's what I understood about the C ++ compiler. My first build uses generic.r.

I will follow your insttructions.

If a can find the options that build and worth it, I intended to make a build daemon, publish the builded material, and extracting a build log, a test log, and a trace of a run. This could help to isolate bugs.

@hostilefork
Copy link
Member

hostilefork commented Mar 23, 2019

I realized that your obsd-build branch was not merged up to date...the last commit it had was January 11th.

I had just seen the most recent date on the commit and thought it must be newer. But when I rebased it, I get the segmentation fault you see.

So now I have a repro case...

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 23, 2019

Yes I rebased master and forget this branch! For my second build test I used a separate clone.

@hostilefork
Copy link
Member

hostilefork commented Mar 23, 2019

Okay, I found the problem! It wasn't very hard to see what was wrong once I looked at the crash in the debugger:

e74dc7d

So that should fix it. Good to find these things--so I'm glad you're making the effort to build it on OpenBSD. And I do think that it's good to stay building under old compilers because it's a good test that you must be keeping dependencies under control (!)

Though I will say...that if you have an interest in C/C++ programming, it looks like OpenBSD may be a painful place to be learning it. If you're an open source purist and want something that's still pretty full-featured as an IDE, I suggest Qt Creator on a Debian install:

https://www.qt.io/download

Would be much easier to do most of your developing/debugging that way, even if you ultimately want an OpenBSD executable for some reason...

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 25, 2019

What do you mean about open source purist?

I was in Debian for about 10 years at home. For 3 years my PC broke down and I went under Windows because I did not want to throw the license forcibly sold with the machine. I use Debian professionally on a daily basis by administering the web infrastructure of my company. I have the project to migrate to OpenBSD instead of Debian because by regularly following their mailing list I found more in common with its developers and their state of mind. At the moment I have three machines under: Debian, OpenBSD, Windows 7. I am rather in the learning of the system and the compilation of ren-c gives me opportunities.

@hostilefork
Copy link
Member

What do you mean about open source purist?

I mean some of the better systems are still commercial or closed, e.g. CLion. There's good and bad about Microsoft Visual Studio, to where I'd call it break-even with Qt Creator.

I am rather in the learning of the system and the compilation of ren-c gives me opportunities.

If you find something you are interested in working on as a project, let me know. But we're really in need of contributions at any level--so one can make improvements without them being huge amounts of code. Right now I would prefer it if we can see code get deleted and maintain the same functionality, vs getting new features...

I use Debian professionally on a daily basis by administering the web infrastructure of my company.

We can use some help with web infrastructure, given the desire to put it online!

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 25, 2019

I find VS to be heavy. I am rather old school. A good editor (EditPad Pro) a grep tool (AstroGrep), a merge tool (WinMerge), a terminal, and I'm happy. I did not really do C / C ++ programming, only occasionally. I have heard a lot of good JetBrains softwares.

I started reformatting all comment by hand in C89 style but you did a lot of change in sources. I do not know if I can send everything at once or if I have to do it file it by file. There is also third-party library code and I do not know whether to keep the code inchanged so that it can easily be updated from upstream.

Deleting code and keeping features inchanged in the same time is a must have.

For web infrastructure, indeed I can help us to be self-hosted.

@hostilefork
Copy link
Member

I started reformatting all comment by hand in C89 style but you did a lot of change in sources.

I don't know what code you are talking about, but the Ren-C sources use // and are not going to change:

https://github.com/metaeducation/ren-c/wiki/On-Building-Ren-C-With-Cpp-Compilers

I much prefer the // style, and they were incorporated into most C compilers long before the C99 standard made it official.

I use C89 comments in generated code files, so that one can notice them and realize "hey, the project doesn't usually use this, I must be looking at generated code...I should not edit here unless I'm in the generator".

If you wished, you could write an automated tool to filter the files pre-build and make C89 comments. PARSE rules could do it. Notice how a patch was made recently:

https://github.com/metaeducation/ren-c/blob/3e965b4a79f870228588af0c2543ef7579734bc6/make/tools/libr3.js-workers-patch.reb

You could do something similar and patch comments when you built, if you thought it necessary. It's not something I would advise spending time on.

@lkppo
Copy link
Collaborator Author

lkppo commented Mar 26, 2019

Ok, I understand why you wanted to do it with PARSE. I left that for someone proefficient with it.

@hostilefork
Copy link
Member

I am rather in the learning of the system and the compilation of ren-c gives me opportunities.

Something to read about that's rather important and actually a big deal when doing low-level C programming:

https://stackoverflow.com/questions/98650/what-is-the-strict-aliasing-rule

One has to understand that to know why some of the things in Ren-C are done the way they are.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants