Windows binary #138
Comments
|
I've had success building a stripped-down babashka in a Windows 10 VM. I don't have a summary ATM of what I did, but one thing that came out of it was: It appears there is a fix on the way for that. Will try to accumulate a summary of the current state in this comment. TLDR: By removing some bits, it appears possible to build a version of babashka that runs on Windows 10. Such bits include some unix-specifc parts (e.g. signal-handing) and SSL. What needs to be done may be different for a Java 8-based build vs a Java 11-based build (e.g. also removed network bits and reflection bits). The required dev environments also appear to differ (Windows 7.1 SDK vs VS 2017 + bits). Longer Version: Java 8-based (Windows 7.1 SDK) build:
Java 11-based (VS 2017 + Visual C++ compilers and libraries for ARM64 and Visual C++ tools for CMake) build: In addition to the Java-8 bits mentioned above:
network portions:
some network portions:
Note: info concerning automated installation of dev bits here: #138 (comment) |
|
Are there considerations that dictate it must be stripped down, or would it be possible to have full functionality? |
|
I'll try to replicate what was involved. One thing that didn't work was the pipe-signal-related functionality. Disabling that was one of the things that I did. |
|
Do you think that might work in a gitbash environment or was it just something that's out of the question if running on windows at all? |
|
I honestly don't know :) IIRC, at least part of the reason for pipe-signal is there is to support the appropriate termination of the sequence-of-processes-connected-by-pipes construct. If you don't use babashka in that kind of situation, may be it will be fine for the most part? |
|
Some other things I removed: from main.clj and the associated part from reflection.json, and from reflection.json. |
|
I hadn't really thought about it, but it does seem both cmd and powershell have pipes. So compatibility there might have specific semantics. I assume gitbash pipes are similar to unix pipes? I don't really know. But gitbash seems to generally work with bash scripts so it must be doing something right on that front. Path libs might be hard... I don't know how that's dealt with in the Java ecosystem. But I assume there's a good way, since Java was meant to generally run everywhere. |
|
I just tried this in git bash: |
|
A good test for the pipe stuff is the following: Without the pipe signal, this chain would hang forever. |
|
It looks messy, but in the vaniall command prompt, I get: |
|
I tried borkdude's command and it is hanging after giving the following output: This was with both git bash and the vanilla command prompt. |
|
For reference, below is what I used for compiling: I think all I did was to adapt some other borkdude (TM) compile.bat + compile scripts and removed the SSL bits. |
|
Below is a diff for reflection.json: |
|
Below is a diff from main.clj: |
|
I think that about covers it as I made those changes to the latest babashka, compiled, and reported those command results above. Haven't tried to port the tests :) |
|
Maybe JDK 11 has better support for the commented out functionality? |
|
That would be nice. I guess I can get graal 19.3.0 Java 11 and give it a try. |
|
Tried with graal 19.3.0 Java 11... Could this be some of that Reflector fun you mentioned in #120? Full details.\script\compile.bat Building bb 0.0.38-SNAPSHOT Compiling babashka.impl.Boolean Compiling babashka.impl.Double Compiling babashka.impl.async Compiling babashka.impl.clojure.core Compiling babashka.impl.clojure.core.server Compiling babashka.impl.clojure.java.io Compiling babashka.impl.clojure.main Compiling babashka.impl.clojure.stacktrace Compiling babashka.impl.conch Compiling babashka.impl.csv Compiling babashka.impl.exceptions Compiling babashka.impl.me.raynes.conch.low-level Compiling babashka.impl.pipe-signal-handler Compiling babashka.impl.socket-repl Compiling babashka.impl.tools.cli Compiling babashka.main Compiling babashka.wait Compiling sci.core Compiling sci.impl.analyzer Compiling sci.impl.destructure Compiling sci.impl.doseq-macro Compiling sci.impl.fns Compiling sci.impl.for-macro Compiling sci.impl.interop Compiling sci.impl.interpreter Compiling sci.impl.main Compiling sci.impl.max-or-throw Compiling sci.impl.namespaces Compiling sci.impl.parser Compiling sci.impl.readers Compiling sci.impl.utils Created C:\Users\user\Desktop\babashka\target\babashka-0.0.38-SNAPSHOT.jar Created C:\Users\user\Desktop\babashka\target\babashka-0.0.38-SNAPSHOT-standalone.jar Warning: the '=' character in program arguments is not fully supported. Make sure that command line arguments using it are wrapped in double quotes. Example: "--vm.Dfoo=bar" |
|
@borkdude Please provide more details regarding where to comment out the use of |
|
@sogaiu That would be here in sci: What we could do is copy the Reflector.java class and distribute it as a separate library that is suited for JDK 11. |
|
Thanks -- though isn't |
|
@sogaiu Let me |
|
@sogaiu Sorry, I think I misread you. Yes, but you can just uneval those expressions. You won't have interop so things like |
|
Thanks. I think I misunderstood too, but I have a better idea now :) |
|
@borkdude Ok, those changes and adding |
|
Searching for At least a couple of people have been assigned to 1870 -- but ATM it appears there is no feedback. |
|
I wil try to disable / remove the socket repl and/or network part of wait. |
|
Ah, I just realized I haven't been using the "recommended" dev environment mentioned in oracle/graal#1852 (comment) |
|
@borkdude I tried removing various network-related looking bits, but I still see the same thing about What I did:
Is there anything obviously missing at this level of description without going into the details of what I did? P.S. I have not changed my dev env. |
|
Will try that, thanks. |
|
Ok, got a bit further -- after examining a bit of what's in the reports directory, I remembered having added the following to compile.bat (native-image options): So I took that out, and now I get: |
|
Found this eclipse-openj9/openj9#1697 |
|
Not really clear yet, but may be this is a place where it will make a difference which dev env is used. |
|
Also found oracle/graal#1850 |
|
I tried installing https://chocolatey.org/packages/visualstudio2017buildtools -- but the build produced (via the developer prompt): |
|
Looking closely at some of the commands for activating the development environment [1], it might be better to try: I'll try that next. [1] remkop mentioned:
|
|
As a side note, remkop mentioned in oracle/graal#1852 (comment):
I don't know if either of the choco commands above cause this to occur. |
|
Unfortunately, the above choco command for the community edition appears to have been insufficient: |
|
The command that remkop mentioned: doesn't work: so perhaps some bits are missing. |
|
The choco command for the community edition does lead to being able to launch the graphical installer "Visual Studio Installer" from which one can do the following:
May be there's some way to automate this? At any rate, after the above sequence completed, the command: is successful! |
|
Found this:
more at: |
|
Successful build and run :) |
|
Thanks to @filipesilva's info on slack, here is a sketch of a command that might be used to automate the installation of vs 2017 and associated bits: The names for what to add were obtained via: which in turn was found via his posted link: https://docs.microsoft.com/en-us/visualstudio/install/workload-and-component-ids?view=vs-2017 May be we want the "-y" flag too. Haven't tested it yet. |
|
I didn't have a clean environment to test from scratch, but I uninstalled once and then ran (in powershell):
Then (in a command prompt): Observed success. Then, ran the build script in the same command prompt window. Observed successful compilation. Finally, successful trivial run of bb.exe. |
|
Github actions Windows runners already contain VS 2019 with the CMake workload, but not the ARM64 one: https://help.github.com/en/actions/automating-your-workflow-with-github-actions/software-installed-on-github-hosted-runners#windows-server-2019 Testing windows builds in those machines should reduce total run time by not having to download everything. |
|
If that works out, it would certainly be nicer :) FWIW, IIUC, there are some opinions among the oracle/graal issues that since the devs seem to be using VS 2017, it might be less problematic to use that version. |
|
As an experiment, I tried a native-image build of babashka using graal 19.3.0 Java 8 and the VS 2017 bits. It didn't work as-is. I'm not sure if it's supposed to work. Full output.\script\compile.bat Building bb 0.0.38-SNAPSHOT Compiling babashka.impl.Boolean Compiling babashka.impl.Double Compiling babashka.impl.async Compiling babashka.impl.clojure.core Compiling babashka.impl.clojure.java.io Compiling babashka.impl.clojure.main Compiling babashka.impl.clojure.stacktrace Compiling babashka.impl.conch Compiling babashka.impl.csv Compiling babashka.impl.exceptions Compiling babashka.impl.me.raynes.conch.low-level Compiling babashka.impl.pipe-signal-handler Compiling babashka.impl.tools.cli Compiling babashka.main Compiling babashka.wait Compiling sci.core Compiling sci.impl.analyzer Compiling sci.impl.destructure Compiling sci.impl.doseq-macro Compiling sci.impl.fns Compiling sci.impl.for-macro Compiling sci.impl.interop Compiling sci.impl.interpreter Compiling sci.impl.main Compiling sci.impl.max-or-throw Compiling sci.impl.namespaces Compiling sci.impl.parser Compiling sci.impl.readers Compiling sci.impl.utils Created C:\Users\user\Desktop\babashka\target\babashka-0.0.38-SNAPSHOT.jar Created C:\Users\user\Desktop\babashka\target\babashka-0.0.38-SNAPSHOT-standalone.jar Warning: the '=' character in program arguments is not fully supported. Make sure that command line arguments using it are wrapped in double quotes. Example: "--vm.Dfoo=bar"Executing [ Running command: CL /MD /Zi /PDBSTRIPPED /FeC:\Users\user\Desktop\babashka\bb.exe C:\Users\user\AppData\Local\Temp\SVM-3419764194800344559\bb.obj C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\ffi.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\libchelper.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\strictmath.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\jvm.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\zip.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\net.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\java.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\nio.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\ffi.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\libchelper.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\strictmath.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\jvm.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\zip.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\net.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\java.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\nio.lib /link /INCREMENTAL:NO /NODEFAULTLIB:LIBCMT /NODEFAULTLIB:OLDNAMES /LIBPATH:C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib /LIBPATH:C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64 advapi32.lib ws2_32.lib secur32.lib iphlpapi.lib Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27034 for x64 cl : Command line warning D9002 : ignoring unknown option '/PDBSTRIPPED' /debug Caused by: java.lang.RuntimeException: host C compiler or linker does not seem to work: java.lang.RuntimeException: returned 2 Running command: CL /MD /Zi /PDBSTRIPPED /FeC:\Users\user\Desktop\babashka\bb.exe C:\Users\user\AppData\Local\Temp\SVM-3419764194800344559\bb.obj C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\ffi.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\libchelper.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\strictmath.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\jvm.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\zip.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\net.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\java.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\nio.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\ffi.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\libchelper.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\strictmath.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64\jvm.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\zip.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\net.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\java.lib C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\nio.lib /link /INCREMENTAL:NO /NODEFAULTLIB:LIBCMT /NODEFAULTLIB:OLDNAMES /LIBPATH:C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib /LIBPATH:C:\Users\user\Desktop\graalvm-ce-java8-19.3.0\jre\lib\svm\clibraries\windows-amd64 advapi32.lib ws2_32.lib secur32.lib iphlpapi.lib Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27034 for x64 cl : Command line warning D9002 : ignoring unknown option '/PDBSTRIPPED' /debug Error: Image build request failed with exit status 1 |
|
So ATM, it looks like for building:
|
|
First experimental Windows build of babashka (bb.exe): See appveyor.yml and script\compile.bat. It is compiled with JDK8 like the linux build. The only thing I needed to work around was oracle/graal#1784. I didn't test it extensively yet. |
|
Downloaded zip and unpacked to folder in PATH, then tried to run all samples in https://github.com/borkdude/babashka#examples. In gitbash, all of them worked as expected except for the last one ( In cmd, some stuff worked but quotes and In powershell, I saw some problems with quotes and In powershell, on a larger folder, I saw a crash as well: |
|
@filipesilva In most (all?) of the above failing examples it's just due to difference in cmd.exe and bash syntax. E.g. <<< is bash syntax for reading a string literal. And single quotes don't work in cmd.exe whereas they do in bash. |
|
@filipesilva It would be interesting to see if the script examples from the gallery work:
And the deps.clj script: |
|
Perhaps everyone is already familiar with the following, but for reference, I found the following helpful in providing hints for quoting differences among multiple "command prompts" / "shells": https://github.com/clojure/tools.deps.alpha/wiki/clj-on-Windows#escaping-quotes |
|
Can you give me example usage of the All scripts worked in gitbash, powershell, and cmd. In gitbash |
|
The ctrl-c behavior has to do with signal handling: oracle/graal#1784. Maybe we'll have to wait for that to be fixed before releasing an official version?
Thanks, that's promising! As for should show you what you can do with it. |
|
@sogaiu I didn't know about that link but wow... everything has a different combination of quotes, double quotes, and escaped quotes.... @borkdude well the ctrl+c problem only happens on gitbash, and when you're using gitbash you already buy into a certain amount of weirdness. Plus it's only relevant for non-exiting commands. IMHO that isn't strong enough of a reason to avoid the release. |
|
The Windows binary has been released for some time now. I'll close this issue and I'll make a new issue for porting the tests. |
Is your feature request related to a problem? Please describe.
It would be nice to also have a windows binary.
Describe the solution you'd like
A windows binary distributed on github, maybe a way to install it too (e.g. a install script like the others).
I'm not sure if there are limitations on how it works, but ideally for me it would work when invoked from cmd, powershell, and gitbash.
Describe alternatives you've considered
I could use WSL and use babashka from there. But that's a fair bit of overhead just to launch it.
The text was updated successfully, but these errors were encountered: