Can you post the CMake file for glktermw?

[WakeRealityDev]

Are you working with this branch? https://github.com/erkyrath/glkterm/tree/widechar - Thank you.

[cspiegel]

I'm using http://www.eblong.com/zarf/glk/glktermw-104.tar.gz, but it does appear that zarf's widechar branch of glkterm is the same thing.

I'll add some CMake files for the various Glk implementations that zarf provides.

[cspiegel]

OK, I've pushed changes which include CMakeLists.txt files for cheapglk, glkterm, glktermw, remglk, and xglk.

[WakeRealityDev]

Thank you! Working good. Some of the version information from Gargoyle isn't actually correct, for example this line says Git is 1.3.3 when actually it's 1.3.5: https://github.com/cspiegel/terps/blob/master/CMakeLists.txt#L169

I'm pretty terrible with Makefiles - and I'd like to also get nanoglk into the fold. Source: https://github.com/BroadcastGames/nanoglk

My first stab at a CMakeList.txt isn't working yet, but posting here in case you have time to take a stab at it ;)

cmake_minimum_required(VERSION 3.1)

# QUESTION? Howto? Should SDL1.2 package requirements go here?
#   nanoglk needs SDL 1.2, SDL 1.2 TTF, SDL 1.2 image
#  find_package(PkgConfig REQUIRED)
#  pkg_check_modules(NCURSESW REQUIRED sdlttf)


add_library(glk
   # All pars of nanoglk, including "misc", as well as the blorb and the
   # dispatching layer.
   nanoglk/main.c nanoglk/event.c nanoglk/window.c
   nanoglk/wintextbuffer.c nanoglk/wintextgrid.c
   nanoglk/wingraphics.c nanoglk/stream.c nanoglk/sound.c
   nanoglk/fileref.c nanoglk/image.c nanoglk/dispatch.c
   nanoglk/blorb.c nanoglk/unsorted.c
   
   # The pars of the "misc" subset of nanoglk.
   misc/misc.c misc/string.c misc/ui.c misc/filesel.c
   misc/conf.c
    
   glk/gi_blorb.c
   glk/gi_dispa.c)

target_link_libraries(glk m)

[cspiegel]

I just added a CMakeLists.txt for nanoglk. A little setup needs to be done, which is documented in that file.

[WakeRealityDev]

Thanks a lot Chris, this is great! nanoglk execution crashed for me on first run as it seems to also depend on TrueType font dejavu being installed. On Ubuntu I had to install package ttf-dejavu - we could add a comment to the Makefile or another find_package dependency (although technically it's runtime and not compile). My hope is it gets some more attention and someone can properly update nanoglk to SDL 2.0. Have a great week.

[WakeRealityDev]

Chris, feel free to ignore me on this, as it's kind of a laundry list of terp issues ;) If someone out there has time, can they build this with glkterm and try out advsys terp? from build directory, try the story data file The Key to Time:

$ ./advsys key.dat

On my Ubuntu 17.04 x64 desktop, this fails with "bad data file", and sometimes Gargoyle itself was giving me this error, but I can't figure out the circumstances. I closed an issue on it as I thought it was my mistake: garglk/garglk#282

[cspiegel]

Advsys as modified for Gargoyle was calling glkunix_stream_open_pathname at an arbitrary point (i.e. outside of glkunix_startup_code; this is not valid in the normal Unix Glk implementations, because glkunix_stream_open_pathname is inherently platform-specific and cannot be called where standard Glk functions are expected. Gargoyle “knows” that this function is always available to it, though, and so it was being used where it's not really supposed to be.

I added a quick hack to advsys to open the file in the startup code.

[WakeRealityDev]

Thanks for the AdvSys fix. In Gargoyle it almost seems like a race condition, as sometimes it works and sometimes it fails. But now it's solid.

Your series of CMake files is much better than the mess I was creating. I've had good success adapting it to the Android NDK cross-compiler to produce executable binaries for all the CPU variations. A key trick so far was to modify the terp function:

function(terp targetraw)
  # rewrite target variable
  set(target "lib_app_${targetraw}.so")

This got Android Studio compiling everything except WITH_BOCFEL and WITH_FROTZ - I need to track down some compile errors with those two. I'll have more time to track those down later today. Thank you again!

[cspiegel]

Thanks for the AdvSys fix. In Gargoyle it almost seems like a race condition, as sometimes it works and sometimes it fails. But now it's solid.

There is a bit of weirdness in Gargoyle interpreters in that the directory is changed before the interpreters have a chance to open game files. I'm not sure of the motivation for this behavior—it was added explicitly for OS X and then for all platforms, meaning there was probably a reason for it. I've wanted to change it, but without knowing why it's there, I'm a bit wary... but the upshot is that if you use a relative path to a game file it may not work. e.g.:

$ gargoyle ../key.dat

results in “bad story file”, but:

$ gargoyle $PWD/../key.dat

results in the game working as intended. It's possible you were bitten by this bug (I have to think it's a bug; I'll take some time to see if I can figure out whether removing the chdir() call has negative consequences).

This got Android Studio compiling everything except WITH_BOCFEL and WITH_FROTZ - I need to track down some compile errors with those two.

Feel free to paste the errors here. I wrote Bocfel so I would probably be able to have some insight into what's going on there; and if it's an issue in Bocfel I'd like to fix it upstream. And we can always patch our Frotz code as well, if it's a bug there.

[WakeRealityDev]

I posted over on Fizmo an invite to get it into this project. It already has all the code to interface with Glk in it's RemGlk branch. It's been actively developed this year, and I think it has a a healthy SDL2 layer that could be the inspiration for a general-purpose SDL2GLK / nanoglk upgrade. chrender/fizmo#23

[WakeRealityDev]

re: @cspiegel "Feel free to paste the errors here."

Here is what I've changed to run in Android Studio 2.3.3 integrated CMake system: WakeRealityDev@4437c5e

Both WITH_BOCFEL and WITH_FROTZ cause the compile/link to fail, all the others seem to run through all the CPU architectures. Here is WITH_BOCFEL and no Frotz:

Starting process 'command '/spot/WorkDev0/devPlace0/android_dev0/sdk/cmake/3.6.4111459/bin/cmake''. Working directory: /spot/WakeReality0/android/IFTerpCrossCompileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines Command: /spot/WorkDev0/devPlace0/android_dev0/sdk/cmake/3.6.4111459/bin/cmake --build /spot/WakeReality0/android/IFTerpCrossCompileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines/.externalNativeBuild/cmake/release/mips64 --target lib_app_bocfel.so
Successfully started process 'command '/spot/WorkDev0/devPlace0/android_dev0/sdk/cmake/3.6.4111459/bin/cmake''
[1/1] Linking C executable /spot/WakeReality0/android/IFTerpCrossCompileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines/src/main/jniLibs/mips64/lib_app_bocfel.so
FAILED: : && /spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/clang  --target=mips64el-none-linux-android --gcc-toolchain=/spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=/spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/sysroot -isystem /spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/sysroot/usr/include/mips64el-linux-android -D__ANDROID_API__=21 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fintegrated-as -Wa,--noexecstack -Wformat -Werror=format-security  -fPIC -O2 -DNDEBUG  -Wl,--exclude-libs,libgcc.a --sysroot /spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/platforms/android-21/arch-mips64 -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -Wl,--gc-sections -Wl,-z,nocopyreloc -pie -fPIE  -fPIE -pie    -flto CMakeFiles/lib_app_bocfel.so.dir/bocfel/blorb.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/branch.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/dict.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/iff.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/io.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/math.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/meta.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/memory.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/objects.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/osdep.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/process.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/random.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/screen.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/sound.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/stack.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/unicode.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/util.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/zoom.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/zterp.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/glkstart.c.o  -o /spot/WakeReality0/android/IFTerpCrossComp
ileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines/src/main/jniLibs/mips64/lib_app_bocfel.so  remglk/libglk.a -lm -lm && :
CMakeFiles/lib_app_bocfel.so.dir/bocfel/blorb.c.o: file not recognized: File format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
:fictionengines:externalNativeBuildRelease FAILED
:fictionengines:externalNativeBuildRelease (Thread[Daemon worker Thread 7,5,main]) completed. Took 0.203 secs.
:app:buildInfoGeneratorDebug (Thread[Daemon worker Thread 7,5,main]) started.
:app:buildInfoGeneratorDebug
Putting task artifact state for task ':app:buildInfoGeneratorDebug' into context took 0.0 secs.
Executing task ':app:buildInfoGeneratorDebug' (up-to-date check took 0.0 secs) due to:
  Task has not declared any outputs.
:app:buildInfoGeneratorDebug (Thread[Daemon worker Thread 7,5,main]) completed. Took 0.004 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':fictionengines:externalNativeBuildRelease'.
> Build command failed.
  Error while executing process /spot/WorkDev0/devPlace0/android_dev0/sdk/cmake/3.6.4111459/bin/cmake with arguments {--build /spot/WakeReality0/android/IFTerpCrossCompileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines/.externalNativeBuild/cmake/release/mips64 --target lib_app_bocfel.so}
  [1/1] Linking C executable /spot/WakeReality0/android/IFTerpCrossCompileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines/src/main/jniLibs/mips64/lib_app_bocfel.so
  FAILED: : && /spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/clang  --target=mips64el-none-linux-android --gcc-toolchain=/spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/toolchains/mips64el-linux-android-4.9/prebuilt/linux-x86_64 --sysroot=/spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/sysroot -isystem /spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/sysroot/usr/include/mips64el-linux-android -D__ANDROID_API__=21 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fintegrated-as -Wa,--noexecstack -Wformat -Werror=format-security  -fPIC -O2 -DNDEBUG  -Wl,--exclude-libs,libgcc.a --sysroot /spot/WorkDev0/devPlace0/android_dev0/sdk/ndk-bundle/platforms/android-21/arch-mips64 -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -Wl,--gc-sections -Wl,-z,nocopyreloc -pie -fPIE  -fPIE -pie    -flto CMakeFiles/lib_app_bocfel.so.dir/bocfel/blorb.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/branch.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/dict.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/iff.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/io.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/math.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/meta.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/memory.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/objects.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/osdep.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/process.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/random.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/screen.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/sound.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/stack.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/unicode.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/util.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/zoom.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/zterp.c.o CMakeFiles/lib_app_bocfel.so.dir/bocfel/glkstart.c.o  -o /spot/WakeReality0/android/IFTerpCrossCompileAndroid/Google_sample_based/loadable_engines_hello-jni_GitHub0/fictionengines/src/main/jniLibs/mips64/lib_app_bocfel.so  remglk/libglk.a -lm -lm && :
  CMakeFiles/lib_app_bocfel.so.dir/bocfel/blorb.c.o: file not recognized: File format not recognized
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  ninja: build stopped: subcommand failed.

[WakeRealityDev]

I know that last post log isn't much, so here is a full log of the compile/build. Note that stderror is appended at the end:

https://gist.github.com/WakeRealityDev/d3efcbc1cd335636f8b49e689bd72929

[cspiegel]

I've never used the Android NDK before so I may be doing something wrong, but I tried something like this on the terpsTemp repository:

cmake .. -DCMAKE_TOOLCHAIN_FILE=/home/chris/programs/android-ndk-r15c/build/cmake/android.toolchain.cmake

I was able to get Frotz to fail to build, but Bocfel built just fine for me. Glulxe didn't build, and a cursory glance looks like remglk claims to support GIDISPATCH_AUTORESTORE_REGISTRY but doesn't; I'll look into that later.

On the Bocfel side, for some reason an object file is being produced for you that is not valid. Just for reference, can you run file on the blorb.c.o file, and then also on one of the other object files created for Bocfel? It'd be interesting to see how/if the file types differ. And also, can you rerun ninja with the -v flag? That will show the commands used to build the source files, so we can take a look and see if, for some reason, blorb.c is being built differently than the other source files...

As for Frotz, its Glk implementation looks to be done by simply redefining many of the functions/types in stdio.h, which fails spectacularly with the Android C library. This probably isn't easily fixable: all the calls to standard I/O functions would have to manually be replaced with the equivalent Glk calls. This would mean even more divergence from upstream than already exists, since upstream does not support Glk.

[WakeRealityDev]

re : "As for Frotz, its Glk implementation looks to be done by simply redefining many of the functions/types in stdio.h, which fails spectacularly with the Android C library. This probably isn't easily fixable: all the calls to standard I/O functions would have to manually be replaced with the equivalent Glk calls. This would mean even more divergence from upstream than already exists, since upstream does not support Glk."

Android NDK has a ton of options on the stdio, so if that's the issue, I can tweak those. it offers static vs. dynamic, gnu vs. others, etc - here is the relvant doc page: https://developer.android.com/ndk/guides/cpp-support.html

The question is - should I change this at the root project level or should we change it only for Frotz? Where should I insert CMake lines and which lines?

[WakeRealityDev]

re: "can you run file on the blorb.c.o file, and then also on one of the other object files created for Bocfel? "

$ cd ./fictionengines/.externalNativeBuild/cmake/release/mips64/CMakeFiles/lib_app_bocfel.so.dir/bocfel/
$ file blorb.c.o
blorb.c.o: LLVM IR bitcode
$ file *
blorb.c.o:    LLVM IR bitcode
branch.c.o:   LLVM IR bitcode
dict.c.o:     LLVM IR bitcode
glkstart.c.o: LLVM IR bitcode
iff.c.o:      LLVM IR bitcode
io.c.o:       LLVM IR bitcode
math.c.o:     LLVM IR bitcode
memory.c.o:   LLVM IR bitcode
meta.c.o:     LLVM IR bitcode
objects.c.o:  LLVM IR bitcode
osdep.c.o:    LLVM IR bitcode
process.c.o:  LLVM IR bitcode
random.c.o:   LLVM IR bitcode
screen.c.o:   LLVM IR bitcode
sound.c.o:    LLVM IR bitcode
stack.c.o:    LLVM IR bitcode
unicode.c.o:  LLVM IR bitcode
util.c.o:     LLVM IR bitcode
zoom.c.o:     LLVM IR bitcode
zterp.c.o:    LLVM IR bitcode
$ ls -l
total 600
-rw-r--r-- 1 adminsag adminsag  12280 Aug 28 21:26 blorb.c.o
-rw-r--r-- 1 adminsag adminsag   6688 Aug 28 21:26 branch.c.o
-rw-r--r-- 1 adminsag adminsag  25976 Aug 28 21:26 dict.c.o
-rw-r--r-- 1 adminsag adminsag   7540 Aug 28 21:26 glkstart.c.o
-rw-r--r-- 1 adminsag adminsag   9480 Aug 28 21:26 iff.c.o
-rw-r--r-- 1 adminsag adminsag  24268 Aug 28 21:26 io.c.o
-rw-r--r-- 1 adminsag adminsag   8252 Aug 28 21:26 math.c.o
-rw-r--r-- 1 adminsag adminsag  15564 Aug 28 21:26 memory.c.o
-rw-r--r-- 1 adminsag adminsag  41516 Aug 28 21:26 meta.c.o
-rw-r--r-- 1 adminsag adminsag  41896 Aug 28 21:26 objects.c.o
-rw-r--r-- 1 adminsag adminsag   7016 Aug 28 21:26 osdep.c.o
-rw-r--r-- 1 adminsag adminsag  45224 Aug 28 21:26 process.c.o
-rw-r--r-- 1 adminsag adminsag  11196 Aug 28 21:26 random.c.o
-rw-r--r-- 1 adminsag adminsag 102808 Aug 28 21:26 screen.c.o
-rw-r--r-- 1 adminsag adminsag   3228 Aug 28 21:26 sound.c.o
-rw-r--r-- 1 adminsag adminsag  97264 Aug 28 21:26 stack.c.o
-rw-r--r-- 1 adminsag adminsag  16932 Aug 28 21:26 unicode.c.o
-rw-r--r-- 1 adminsag adminsag  19072 Aug 28 21:26 util.c.o
-rw-r--r-- 1 adminsag adminsag   5312 Aug 28 21:26 zoom.c.o
-rw-r--r-- 1 adminsag adminsag  70156 Aug 28 21:26 zterp.c.o

I'm honestly not that worried about this issue on the short term. As I think I can build two working Z interpreters for Android using the legacy gcc cross-compiler system that Android NDK used to offer https://github.com/WakeRealityDev/IFTerpCrossCompileAndroid . I'm really happy to have a well-organized and desktop+Android system with CMake and clang - and solving these two terps isn't a showstopper for me - as they are both Z-Machine and we have multiple to pick from. nitfol works!

[WakeRealityDev]

CONFIRMED: I do have a working Android.mk (legacy NDK build system / Makefile system) that can cross-compile Bocfel on Android. I DO NOTICE SOMETHING! I spent hours on that and found out that "include 'limits.h'" on the LOCAL_CFLAGS solved problems that other Gargoyle terps did not require.

# -include "limits.h" fixes SIZE_MAX references
LOCAL_CFLAGS    := -std=gnu11 -DZTERP_UNIX -DZTERP_GLK -DGARGLK \
                     -include "limits.h" \
                     -g

Can we add some similar include to CMake side, where and what?

As for Frotz - seems I never got that far and have no notes of getting it working on Android - and since I had both nitfol and bocfel working on all the CPU variations - I moved forward without frotz.

[WakeRealityDev]

Fresh look this morning, I discovered a clue: If I tell the NDK to cross-compile only one time, only x86 CPU, it works for WITH_BOCFEL. - is the CMake setting some var that the 2nd cycle through is causing problems?

EDIT: it seems like only one CPU, picking 'armeabi-v7a' exclusive, also fails. Still trying to figure it out.

[cspiegel]

I'm unable to reproduce the issue; what's the exact cmake invocation you're using?

If unconditionally including limits.h in each file fixes the build issue... Well that's weird, because I'm not sure how its omission could cause the object file to be an unrecognized format, and I'm also pretty sure all files which make use of the constants in limits.h are including that file. If you have a little time, it'd be worthwhile to see whether adding #include <limits.h> at the top of each file instead of passing it via -include solves the issue. I want to say it's a toolchain problem, but if Bocfel is missing a limits.h somewhere I'd want to fix it.

[WakeRealityDev]

While I'm thinking about it: can you make tads terp like the others, as a WITH_TADS option?

[WakeRealityDev]

I think I identified the problem. https://github.com/cspiegel/terps/blob/master/cmake/Modules/Compilers.cmake#L31

target_compile_options(${target} PUBLIC -flto)

I comment that out, WITH_BOCFEL compiles fine for arm CPU. However, there is still some weirdness - I am making sure to completely delete my build directories each run, as it isn't behaving consistent.

If I purge my build directory, and do only Bocfel, I can compile: 'arm64-v8a', 'x86' in one run (two cross-compile runs).

[cspiegel]

Great, thank you for tracking that down. The failure with LTO in Bocfel is due to a Clang bug (see https://bugs.llvm.org/show_bug.cgi?id=32155 and android/ndk#350, for example). The problem is that the linker doesn't understand the -Os flag when dealing with LTO, but Clang still passes it. One solution is to modify android.toolchain.cmake and change -Os to -O2. As an alternative, I've added an ENABLE_LTO flag which allows LTO to be disabled.

WITH_TADS already exists, although it's defined in tads/CMakeLists.txt instead of the top-level one.

[WakeRealityDev]

Chris: Thank you for all the labor of validating issues. Above and beyond your own personal projects. One tiny -OB arg in makin' C coding for 7 different CPU variations and billions of varied devices can result in months of discouraging results that have erratic failures like Bocfel showed. It's really supportive to have collaborative effort into the time-travel C code of decades past into the modern code editor. It's an adventure to the adventure.