Bazelify the GMP Precision Arithmetic Library #2

rauljordan · 2018-11-16T19:59:50Z

Hi all,

A critical component of this library is the GNU Multiple Precision Arithmetic Library written in C used for most underlying cryptographic calculations that are part of BLS. A major problem with this is that most architectures do not ship with GMP installed globally, and therefore any user wishing to use go-bls in their project will need to pollute their global namespace with libgmp.

We have opted for using the Bazel build system created by Google for dependency management and packaging of this repository for use in other projects. In particular, go-bls will be a critical component of the Prysm client created by Prysmatic Labs for Ethereum 2.0. Prysm uses the Bazel build tool extensively and ideally we'd want to import go-bls via Bazel as well. The problem with this is that GMP is not included as a Bazel dependency of go-bls, and therefore the build will fail due not finding the library.

A solution to this would be to convert lib GMP into a Bazel project following the guidelines for C/C++ projects in the Bazel documentation here. This is fairly extensive effort as GMP is a very big library containing multiple pieces. Tackling this issue would allow Prysm and any other Bazel-enabled project to import go-bls without requiring users to have GMP installed on their machine.

We are requesting to bounty this project out for motivated individuals to make this happen. This is a high priority for us at Prysmatic Labs.

Starting Point

This issue created by @prestonvanloon is a great starting point for someone tackling this: unable to gen_rule as part of http_archive.

Preston began by using http_archive to pull the libgmp official repo and begin creating a bazel BUILD file for the system directly.

gitcoinbot · 2018-11-17T20:08:35Z

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

This issue now has a funding of 1000.0 DAI (1000.0 USD @ $1.0/DAI) attached to it as part of the Ethereum Foundation fund.

If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
Want to chip in? Add your own contribution here.
Questions? Checkout Gitcoin Help or the Gitcoin Slack
$54,603.34 more funded OSS Work available on the Gitcoin Issue Explorer

gitcoinbot · 2018-11-17T21:01:05Z

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 3 weeks, 1 day ago.
Please review their action plans below:

1) pepeinvest has applied to start work (Funders only: approve worker | reject worker).

Creative Bussines developers And finance LTd

Learn more on the Gitcoin Issue Details page.

2) pacamara has applied to start work (Funders only: approve worker | reject worker).

Hi! I'd like to start work on this. Action plan: 1. Install Bazel. 2. Write appropriate rules for lib GMP. 3. Test on Ubuntu, OSX, and possibly Windows (if required).

Learn more on the Gitcoin Issue Details page.

3) robin-thomas has been approved to start work.

Download GMP tar file using new_http_archive() instead of http_archive
Create the BUILD file to convert the library into a bazel repo
Add the above dependency to go-bls library

Learn more on the Gitcoin Issue Details page.

spm32 · 2018-11-21T02:43:52Z

Hey @jakerockland you're approved but please let us know ASAP if you don't think you'll be able to do it :)

samparsky · 2018-11-21T07:02:11Z

@jakerockland If you want we can possibly work on together on this & maybe share the bounty?

jakerockland · 2018-11-21T07:28:12Z

@rauljordan @ceresstation To make sure I'm understanding the issue correctly, is the end deliverable for this something along the lines of a new repository containing the source for GMP, with a Bazel build file that allows the GMP libraries to be built with Bazel instead of Make?

jakerockland · 2018-11-21T07:28:29Z

@samparsky I'm open to collaborating if there is a sensible way to split things up, what did you have in mind?

prestonvanloon · 2018-11-21T19:36:43Z

@jakerockland Yes. This should be built entirely using Bazel as the build tool.

It should build on:

x86_64
Mac OS X
Windows
arm64
And any other relevant cpu architecture.

jakerockland · 2018-11-21T21:40:24Z

@prestonvanloon Thank you for the clarification. Given the high priority of this and what else I have in the pipeline, after taking some time to scope this out last night and realize how much of a behemoth/custom configuration scripts there are with GMP and confirming now that I understood properly, I think that it may be better to let someone else take on this bounty @ceresstation.

I've spend a good amount of time today trying to get things to work with this approach, to no avail bazel-contrib/rules_go#1233. Have you seen this issue @prestonvanloon?

spm32 · 2018-11-25T01:53:28Z

Fair @jakerockland, @samparsky are you still up for taking it on? Do you feel confident you're able to do it?

samparsky · 2018-11-26T07:59:44Z

@ceresstation Started work on it, will let you know my progress by tomorrow

samparsky · 2018-11-27T07:41:24Z

@jakerockland @ceresstation I have been able to get it to build successfully using Bazel on MacOSX. Currently testing for ubuntu using a docker image.
For windows its recommended in the gmplib docs (https://cs.nyu.edu/~exact/core/gmp/index.html) to use a Linux environment Cygwin or MinGW

rauljordan · 2018-11-27T20:52:57Z

That's awesome, thanks for the update @samparsky!

samparsky · 2018-11-28T13:47:58Z

@prestonvanloon I have been able to get it to build on using Bazel on Ubuntu & MacOSX.

But I am facing an issue I am able to get the build to work properly on MacOSX with Bazel sandbox mode off only & on Linux it works with sandbox mode on only.

I think it has something to do with the way the underlying filesystem works in these different OSes.

You can check out the details here
https://github.com/samparsky/bazel-gmp

The only difference between the two folders is local=0 or 1 in the gmplib.BUILD file.
In the mac folder its local=1 which means sandbox mode is off & in linux folder its local=0 which means sandbox mode is on.

prestonvanloon · 2018-11-28T14:31:17Z

@samparsky I took a look at your repository. It looks like your implementation is a genrule calling configure and make.

The goal of this task is to use bazel to build the library, not make. Additionally, you shouldn’t require users to install any system dependencies. They should be able to have a working bazel installation and run bazel build //... without additional steps.

I realize this issue might have been misleading because I attempted to use the genrule approach as a bandaid fix for building go-bls without gmp installed on the system. Sorry about the confusion.

gitcoinbot · 2018-12-06T17:16:18Z

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

@samparsky due to inactivity, we have escalated this issue to Gitcoin's moderation team. Let us know if you believe this has been done in error!

reminder (3 days)
escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

samparsky · 2018-12-06T17:21:02Z

@prestonvanloon I tried building it manually couldn't get it to work. Also emailed the GMPlib mailing list to get any pointers couldn't get any positive response

robin-thomas · 2018-12-13T05:37:15Z

@prestonvanloon If it's not an issue, I would like to work on it. As I understand, you need to download the GMP library, and use bazel to build it.

prestonvanloon · 2018-12-15T00:17:55Z

@robin-thomas sounds good to me. As long as bazel is the tool to actually build it. (Not using bazel as a wrapper for make or similar)

prestonvanloon · 2018-12-15T00:18:38Z

@ceresstation can you help transfer this issue to @robin-thomas please?

robin-thomas · 2018-12-16T01:47:57Z

@prestonvanloon @rauljordan Could you take a look at: https://github.com/robin-thomas/bazelify-gmp?

Its still WIP, but to give you some idea.

prestonvanloon · 2018-12-16T16:35:08Z

Looking great @robin-thomas!

robin-thomas · 2018-12-16T21:18:57Z

@prestonvanloon Thanks! I'll continue working on it. Still loads more to be done.

@ceresstation Can you kindly transfer this issue to me? Thanks.

robin-thomas · 2018-12-18T15:25:08Z

@prestonvanloon, @rauljordan. Wanted to give you an update.

Many of the *.h, *.c files are generated on the fly by executing the configure script, many of which are not even known during the analysis stage of Bazel. Bazel requires these names before the build even starts! Related to how Bazel constructs its dependency graph.

Anyway, the only solution here is to zip all those files into one file with a fixed name that can be passed to Bazel. But then again, there is no native support to pass that zip file as the input to cc_* rules for compilation. The only way around it is using Tree Artifacts, and it's a bit of hack to get it working and compiled into a library.

The configure script also generates loads of *.asm files based on the host architecture. It cannot be passed to Bazel's cc_rules, as Bazel already assumes *.asm files to be assembly code generated by the compiler. bazelbuild/bazel#4856. But that's not the case here, and we need those files to be processed. The only way to do that is to use m4 to process these files into .s files which can then be given to Bazel's cc- rules (its placed in the same zip file above).

Another issue is how GMP compiles many of these generated *.c files. They require macros to be passed to the compiler, which is different for every single file. Else the file cannot be compiled. This may not have been much of an issue, but with Tree Artifacts, this seems a bit hard to solve. And I'm currently trying to solve this problem.

https://github.com/robin-thomas/bazelify-gmp

prestonvanloon · 2018-12-19T11:26:24Z

Thanks for the update @robin-thomas! Can select help with choosing the correct files based on the architecture?

robin-thomas · 2018-12-19T11:35:23Z

@prestonvanloon No. That select is a bit different. It's like a dictionary lookup based on a key, where the key can be architecture or some other parameters, but the value stored in the dictionary should be pre-defined. That's not the case here since we don't know the contents of the dictionary beforehand.

Anyway, I think I might have found another way to tackle it. So trying that.

robin-thomas · 2018-12-19T13:50:31Z

Seems like I found a small bug in Bazel's code: bazelbuild/bazel#6963

Maybe I'll wait for someone from their dev team to reply there.

prestonvanloon · 2018-12-20T02:20:12Z

Sounds good, that certainly looks like a bug to me.

rauljordan · 2019-01-01T19:49:38Z

Hey @robin-thomas tried it out on a fresh windows configuration and received this:

ERROR: C:/users/maiji/downloads/go-bls/BUILD.bazel:3:1: in _cgo_collect_info rule //:go_default_library%linux_386%cgo_info:
Traceback (most recent call last):
        File "C:/users/maiji/downloads/go-bls/BUILD.bazel", line 3
                _cgo_collect_info(name = 'go_default_library%linux_386%cgo_info')
        File "C:/users/maiji/_bazel_maiji/tufpbscd/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 414, in _cgo_collect_info_impl
                _CgoInfo(orig_srcs = ctx.files.srcs, transf..., <6 more arguments>)
        File "C:/users/maiji/_bazel_maiji/tufpbscd/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 421, in _CgoInfo
                _select_archives(ctx.attr.libs)
        File "C:/users/maiji/_bazel_maiji/tufpbscd/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 134, in _select_archives
                fail("cc_library(s) did not produce a...")
cc_library(s) did not produce any files
ERROR: Analysis of target '//:go_default_test' failed; build aborted: Analysis of target '//:go_default_library%linux_386%cgo_info' failed; build aborted
INFO: Elapsed time: 436.371s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (38 packages loaded, 7434 targets configured)
FAILED: Build did NOT complete successfully (38 packages loaded, 7434 targets configured)

However I'm not convinced this is an issue with the project but perhaps with how I set everything up - perhaps there was some other system dep I was missing

robin-thomas · 2019-01-01T22:02:58Z

@rauljordan Were you able to build go-bls in Windows before I made my first PR?

Maybe checkout to commit ed485f9efa247225c65b29c1ee00b5b4590fdf38 and run a clean Bazel build (but modify it to use @herumi_mcl to be built in a cc_library rather than from the mcl directory inside go-bls)?

robin-thomas · 2019-01-02T22:37:18Z

@prestonvanloon Can I assume its working on Linux and Mac?

rauljordan · 2019-01-03T06:57:20Z

Hi @robin-thomas, confirmed it works on Linux and Mac. Waiting on a windows confirmation as I no longer have access to a windows machine to attempt. Perhaps @IvanTheGreatDev can help with this?

robin-thomas · 2019-01-03T07:04:32Z

@rauljordan That's great!
@IvanTheGreatDev Waiting to hear from you.

rauljordan · 2019-01-03T19:59:24Z

Hey @robin-thomas tried on another windows machine and received the same error. Checked out to that commit and then received this Java error

INFO: Invocation ID: e825c8f2-4267-4166-b7c9-1e5db3fae0f5
Loading:
Loading: 0 packages loaded
Analyzing: 3 targets (1 packages loaded, 0 targets configured)
Internal error thrown during build. Printing stack trace: java.lang.RuntimeException: Unrecoverable error while evaluating node '//:go_default_library%linux_386%_cgo_.o BuildConfigurationValue.Key[1ad0148498d0d593acb92abfdb3367c7] false' (requested by nodes '//:go_default_library%linux_386%cgo_import BuildConfigurationValue.Key[1ad0148498d0d593acb92abfdb3367c7] false')
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:499)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:368)
        at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(Unknown Source

Sorry this is taking so long to debug - I know also @JerryFireman has windows and can try this out.

robin-thomas · 2019-01-03T21:34:42Z

@rauljordan That's weird. That commit was before I even started working on gmp! So you had never built go-bls on Windows before?

robin-thomas · 2019-01-03T21:55:08Z

@JerryFireman @IvanTheGreatDev Have either of you built go-bls on Windows before?

robin-thomas · 2019-01-03T22:23:22Z

@rauljordan Regarding the first error, I had a look into cgo code.
https://github.com/bazelbuild/rules_go/blob/master/go/private/rules/cgo.bzl#L134

    """Selects a single archive from a list of files produced by a
    static cc_library.
    In some configurations, cc_library can produce multiple files, and the
    order isn't guaranteed, so we can't simply pick the first one.
    """

    # list of file extensions in descending order or preference.
    exts = [".pic.lo", ".lo", ".a"]     <--- maybe need a windows specific extension here
    for ext in exts:
        for f in as_iterable(files):
            if f.basename.endswith(ext):
                return f

Windows static libraries have an extension *.lib (maybe Bazel produces *.a libraries on Windows, but I don't know. Can someone check this out?). If that's the case, then cc_library won't be able to find the library. Hence the error.

robin-thomas · 2019-01-03T23:43:40Z

@rauljordan Also, can you share the details about the Windows setup? What's the compiler used? If its MSVC, then cgo doesn't have support for it.

rauljordan · 2019-01-03T23:46:18Z

Hey Robin, it is indeed MSVC. That must be the issue in that case. @houjieth agreed to try it oit on windows this evening as well.

…

On Thu, Jan 3 2019 at 5:43 PM, < ***@***.*** > wrote: @rauljordan ( https://github.com/rauljordan ) Also, can you share the details about the Windows setup? What's the compiler used? If its MSVC, then cgo doesn't have support for it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( #2 (comment) ) , or mute the thread ( https://github.com/notifications/unsubscribe-auth/AFUIPUUTUZFf9O6-V7KjbpxYO6FVcWO2ks5u_pWtgaJpZM4Ym4L_ ).

robin-thomas · 2019-01-03T23:55:27Z

@rauljordan Yup. That should be the issue then.
cgo does support MSYS, but I don't know to what extent. cgo support for Windows seems a bit poor.

So you never tested on Windows before?

@houjieth Try MSYS + gcc. Or maybe Git Bash (not sure whether a compiler is built into it)

robin-thomas · 2019-01-04T05:18:02Z

@houjieth Any updates?

houjieth · 2019-01-04T10:04:05Z

I'm using native bazel on windows (no cygwin, no mingw, no msys, no other simulated unix environment such as WSL).

Here's my output

C:\Users\houji\go\src\github.com\prysmaticlabs\go-bls>bazel build //...
Starting local Bazel server and connecting to it...
INFO: Repository rule 'io_bazel_rules_m4' returned: {"remote": "https://github.com/jmillikin/rules_m4", "commit": "2bf69df77dfb6b3ba6b7fc95c304b0dc279375bc", "shallow_since": "2018-12-22", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "io_bazel_rules_m4"}DEBUG: C:/users/houji/_bazel_houji/6dp3edvb/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: 'BAZEL_VC' is not set, start looking for the latest Visual C++ installed.
DEBUG: C:/users/houji/_bazel_houji/6dp3edvb/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: Looking for VS%VERSION%COMNTOOLS environment variables, eg. VS140COMNTOOLS
DEBUG: C:/users/houji/_bazel_houji/6dp3edvb/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: Looking for Visual C++ through registry
DEBUG: C:/users/houji/_bazel_houji/6dp3edvb/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: Looking for default Visual C++ installation directory
DEBUG: C:/users/houji/_bazel_houji/6dp3edvb/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: 'PROGRAMFILES(X86)' environment variable is not set, using 'C:\Program Files (x86)' as default
DEBUG: C:/users/houji/_bazel_houji/6dp3edvb/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:115:5:
Auto-Configuration Warning: Visual C++ build tools found at C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC
INFO: SHA256 (https://codeload.github.com/golang/tools/zip/3e7aa9e59977626dc60433e9aeadf1bb63d28295) = 2384fa91351a7414b643c5230422ce45f5aa2be8a82727609afd4e64e6973a30
INFO: Repository rule 'bazelify_gmp' returned: {"remote": "https://github.com/robin-thomas/bazelify-gmp", "commit": "98ccf38c714ccd5527abdb46c16095fa1f622070", "shallow_since": "2018-12-24", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "bazelify_gmp"}
INFO: Repository rule 'herumi_mcl' returned: {"remote": "https://github.com/prysmaticlabs/mcl", "commit": "b90f1fbfea78c47012fcbc9243a616e654794421", "shallow_since": "2018-12-27", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "herumi_mcl"}
INFO: Repository rule 'boringssl' returned: {"remote": "https://github.com/google/boringssl", "commit": "fafc4482e85c09e7af5f71b2eb287b73ccd1020a", "shallow_since": "2018-12-21", "init_submodules": False, "verbose": False, "strip_prefix": "", "patches": [], "patch_tool": "patch", "patch_args": ["-p0"], "patch_cmds": [], "name": "boringssl"}
ERROR: C:/users/houji/go/src/github.com/prysmaticlabs/go-bls/BUILD.bazel:3:1: in _cgo_collect_info rule //:go_default_library%android_amd64%cgo_info:
Traceback (most recent call last):
        File "C:/users/houji/go/src/github.com/prysmaticlabs/go-bls/BUILD.bazel", line 3
                _cgo_collect_info(name = 'go_default_library%android_amd64%cgo_info')
        File "C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 414, in _cgo_collect_info_impl
                _CgoInfo(orig_srcs = ctx.files.srcs, transf..., <6 more arguments>)
        File "C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 421, in _CgoInfo
                _select_archives(ctx.attr.libs)
        File "C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 134, in _select_archives
                fail("cc_library(s) did not produce a...")
cc_library(s) did not produce any files
ERROR: Analysis of target '//:go_default_test' failed; build aborted: Analysis of target '//:go_default_library%android_amd64%cgo_info' failed; build aborted
INFO: Elapsed time: 433.229s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (35 packages loaded, 7453 targets configured)

I'm going to try MinGW + MSYS next.

robin-thomas · 2019-01-04T10:17:00Z

@houjieth From the log, it seems to be using MSVC. If that's the case, then the error is expected.

houjieth · 2019-01-04T11:02:29Z

Tried on MSYS2, got the same error. How can I enforce it to use gcc instead of MSVC? I'm really not familiar with MinGW and MSYS2...

robin-thomas · 2019-01-04T11:08:07Z

Are you able to run the below command?

$ bazel build //... --compiler=mingw-gcc

@houjieth Is there somewhere we can chat?

Also run the below command, and say whats the output:

$ bazel build @bazelify_gmp//:gmp

houjieth · 2019-01-04T12:45:52Z

Now I'm using MSYS2 now without MSVC.

houji@DESKTOP-O5H7M9A MSYS /c/Users/houji/go/src/github.com/prysmaticlabs/go-bls
$ bazel build //...
INFO: Invocation ID: 4ce4bc40-5265-454a-b469-00a9509a87e9
Loading:
Loading: 0 packages loaded
Analyzing: 3 targets (0 packages loaded, 0 targets configured)
ERROR: C:/users/houji/go/src/github.com/prysmaticlabs/go-bls/BUILD.bazel:3:1: in _cgo_collect_info rule //:go_default_library%linux_mipsle%cgo_info:
Traceback (most recent call last):
        File "C:/users/houji/go/src/github.com/prysmaticlabs/go-bls/BUILD.bazel", line 3
                _cgo_collect_info(name = 'go_default_library%linux_mipsle%cgo_info')
        File "C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 414, in _cgo_collect_info_impl
                _CgoInfo(orig_srcs = ctx.files.srcs, transf..., <6 more arguments>)
        File "C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 421, in _CgoInfo
                _select_archives(ctx.attr.libs)
        File "C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 134, in _select_archives
                fail("cc_library(s) did not produce a...")
cc_library(s) did not produce any files
ERROR: Analysis of target '//:go_default_test' failed; build aborted: Analysis of target '//:go_default_library%linux_mipsle%cgo_info' failed; build aborted
INFO: Elapsed time: 0.592s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

houji@DESKTOP-O5H7M9A MSYS /c/Users/houji/go/src/github.com/prysmaticlabs/go-bls
$ bazel build @bazelify_gmp//:gmp
INFO: Invocation ID: 18ac6ddd-788d-4f38-b1f5-edd31178d40d
Loading:
Loading: 0 packages loaded
Analyzing: target @bazelify_gmp//:gmp (0 packages loaded, 0 targets configured)
INFO: Analysed target @bazelify_gmp//:gmp (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 21] no action
ERROR: C:/users/houji/_bazel_houji/6dp3edvb/external/m4_v1.4.18/BUILD.bazel:15:1: C++ compilation of rule '@m4_v1.4.18//:m4_lib' failed (Exit 2)
c:\users\houji\_bazel_houji\6dp3edvb\execroot\__main__\external\m4_v1.4.18\lib\freadahead.h(41): error C2061: syntax error: identifier '__attribute__'
c:\users\houji\_bazel_houji\6dp3edvb\execroot\__main__\external\m4_v1.4.18\lib\freadahead.h(41): error C2059: syntax error: ';'
Target @bazelify_gmp//:gmp failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.788s, Critical Path: 0.70s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

If add --compiler=mingw-gcc

houji@DESKTOP-O5H7M9A MSYS /c/Users/houji/go/src/github.com/prysmaticlabs/go-bls
$ bazel build //... --compiler=mingw-gcc
INFO: Invocation ID: dc0ee7fe-a156-442e-b322-d57dca3822ed
Loading:
Loading: 0 packages loaded
INFO: Build option --compiler has changed, discarding analysis cache.
Analyzing: 3 targets (0 packages loaded, 0 targets configured)
INFO: Analysed 3 targets (0 packages loaded, 7846 targets configured).
INFO: Found 3 targets...
[4 / 16] [-----] BazelWorkspaceStatusAction stable-status.txt
ERROR: C:/users/houji/_bazel_houji/6dp3edvb/external/io_bazel_rules_m4/m4/internal/BUILD:1:1: C++ compilation of rule '@io_bazel_rules_m4//m4/internal:capture_stdout' failed (Exit 1)
external/io_bazel_rules_m4/m4/internal/capture_stdout.c: In function 'main':
external/io_bazel_rules_m4/m4/internal/capture_stdout.c:31:48: error: 'S_IRUSR' undeclared (first use in this function)
     out_fd = open(argv[1], O_CREAT | O_WRONLY, S_IRUSR | S_IWUSR);
                                                ^~~~~~~
external/io_bazel_rules_m4/m4/internal/capture_stdout.c:31:48: note: each undeclared identifier is reported only once for each function it appears in
external/io_bazel_rules_m4/m4/internal/capture_stdout.c:31:58: error: 'S_IWUSR' undeclared (first use in this function)
     out_fd = open(argv[1], O_CREAT | O_WRONLY, S_IRUSR | S_IWUSR);
                                                          ^~~~~~~
INFO: Elapsed time: 2.543s, Critical Path: 0.37s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

houji@DESKTOP-O5H7M9A MSYS /c/Users/houji/go/src/github.com/prysmaticlabs/go-bls
$ bazel build @bazelify_gmp//:gmp --compiler=mingw-gcc
INFO: Invocation ID: 768ff9c7-3663-4166-a1a0-24d2444af630
Loading:
Loading: 0 packages loaded
Analyzing: target @bazelify_gmp//:gmp (0 packages loaded, 0 targets configured)
INFO: Analysed target @bazelify_gmp//:gmp (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
[0 / 17] no action
ERROR: C:/users/houji/_bazel_houji/6dp3edvb/external/m4_v1.4.18/BUILD.bazel:15:1: C++ compilation of rule '@m4_v1.4.18//:m4_lib' failed (Exit 1)
In file included from external/m4_v1.4.18/lib/regex.c:68:
external/m4_v1.4.18/lib/regex_internal.h:29:10: fatal error: langinfo.h: No such file or directory
 #include <langinfo.h>
          ^~~~~~~~~~~~
compilation terminated.
Target @bazelify_gmp//:gmp failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.267s, Critical Path: 0.21s
INFO: 0 processes.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

robin-thomas · 2019-01-04T12:56:12Z

Thanks for the logs @houjieth!
Can you check whether the Windows platform has the m4 binary in PATH?

@prestonvanloon It seems like rules_m4 doesn't work on Windows. So what to do now?

rauljordan · 2019-01-04T23:42:08Z

Hey @robin-thomas, m4 is here on sourceforge http://gnuwin32.sourceforge.net/packages/m4.htm, and perhaps we can use bazel platform selection to download the right distribution? It seems to be downloading the linux version with the current configuation. @prestonvanloon suggested this.

robin-thomas · 2019-01-07T15:31:40Z

@rauljordan @prestonvanloon I had fixed all the m4 issues, and other issues that came later. It now builds GMP library fine. But herumi_mcl build fails. Have you tried building go_bls or herumi_mcl on windows before?

rauljordan · 2019-01-08T16:54:38Z

@houjieth can you give this another try?

0xKiwi · 2019-01-10T09:31:31Z

ivanthegreat@DESKTOP-T8PCO20:/mnt/n/go-bls$ bazel build ...
Starting local Bazel server and connecting to it...
INFO: Analysed 3 targets (37 packages loaded).
INFO: Found 3 targets...
INFO: From Compiling external/m4_v1.4.18/lib/c-stack.c [for host]:
external/m4_v1.4.18/lib/c-stack.c:107:1: warning: 'die' defined but not used [-Wunused-function]
 die (int signo)
 ^~~
INFO: Elapsed time: 161.439s, Critical Path: 138.57s
INFO: 669 processes: 2 local, 667 processwrapper-sandbox.
INFO: Build completed successfully, 678 total actions

Builds on windows!

rauljordan · 2019-01-10T10:04:53Z

The above works on Hyper, but not on MSYS. However, this is good enough for us as a lot of work has already been put in. Allowing for support all across the board including MSYS can be a separate PR or bounty.

rauljordan · 2019-01-10T10:07:34Z

@ceresstation we are happy with the bounty - I can edit the README post merge to include details about building on Windows.

spm32 · 2019-01-10T16:02:44Z

Fantastic to hear @rauljordan, thanks for all of your help @robin-thomas!

gitcoinbot · 2019-01-10T16:05:06Z

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

The funding of 1000.0 DAI (1000.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @robin-thomas.

Learn more on the Gitcoin Issue Details page
Questions? Checkout Gitcoin Help or the Gitcoin Slack
$42,781.51 more funded OSS Work available on the Gitcoin Issue Explorer

luxe · 2019-01-10T16:27:34Z

Just to clarify, is the single bazelify_gmp all that is necessary to start using gmp in bazel projects?

git_repository(
    name = "bazelify_gmp",
    remote = "https://github.com/robin-thomas/bazelify-gmp",
    commit = "bb4881b35e6864c90493980d035e1d984cafd093",
)

Or is it also required to bring in the m4 rules?

git_repository(
    name = "io_bazel_rules_m4",
    remote = "https://github.com/jmillikin/rules_m4",
    commit = "2bf69df77dfb6b3ba6b7fc95c304b0dc279375bc",
)
load("@io_bazel_rules_m4//:m4.bzl", "m4_register_toolchains")

m4 is a dependency of gmp, but I'm not sure if the bazelify-gmp is self-sufficient to use alone. Thanks.

prestonvanloon · 2019-01-10T16:46:38Z

@luxe Yes, I think you would need to install the transitive dependencies.

rauljordan added good for bounty priority: high labels Nov 16, 2018

prestonvanloon mentioned this issue Dec 12, 2018

Add GNU Multiple Precision library as bazel dep prysmaticlabs/pbc#1

Open

prestonvanloon closed this as completed Jan 10, 2019

Bazelify the GMP Precision Arithmetic Library #2

Bazelify the GMP Precision Arithmetic Library #2

Comments

rauljordan commented Nov 16, 2018 • edited Loading

Starting Point

gitcoinbot commented Nov 17, 2018

gitcoinbot commented Nov 17, 2018 • edited Loading

spm32 commented Nov 21, 2018

samparsky commented Nov 21, 2018

jakerockland commented Nov 21, 2018

jakerockland commented Nov 21, 2018

prestonvanloon commented Nov 21, 2018

jakerockland commented Nov 21, 2018 • edited Loading

spm32 commented Nov 25, 2018

samparsky commented Nov 26, 2018

samparsky commented Nov 27, 2018 • edited Loading

rauljordan commented Nov 27, 2018

samparsky commented Nov 28, 2018 • edited Loading

prestonvanloon commented Nov 28, 2018 • edited Loading

gitcoinbot commented Dec 6, 2018

samparsky commented Dec 6, 2018 • edited Loading

robin-thomas commented Dec 13, 2018

prestonvanloon commented Dec 15, 2018

prestonvanloon commented Dec 15, 2018

robin-thomas commented Dec 16, 2018 • edited Loading

prestonvanloon commented Dec 16, 2018

robin-thomas commented Dec 16, 2018

robin-thomas commented Dec 18, 2018 • edited Loading

prestonvanloon commented Dec 19, 2018

robin-thomas commented Dec 19, 2018

robin-thomas commented Dec 19, 2018

prestonvanloon commented Dec 20, 2018

rauljordan commented Jan 1, 2019

robin-thomas commented Jan 1, 2019 • edited Loading

robin-thomas commented Jan 2, 2019 • edited Loading

rauljordan commented Jan 3, 2019

robin-thomas commented Jan 3, 2019

rauljordan commented Jan 3, 2019

robin-thomas commented Jan 3, 2019 • edited Loading

robin-thomas commented Jan 3, 2019 • edited Loading

robin-thomas commented Jan 3, 2019 • edited Loading

robin-thomas commented Jan 3, 2019

rauljordan commented Jan 3, 2019 via email

robin-thomas commented Jan 3, 2019 • edited Loading

robin-thomas commented Jan 4, 2019

houjieth commented Jan 4, 2019 • edited Loading

robin-thomas commented Jan 4, 2019

houjieth commented Jan 4, 2019 • edited Loading

robin-thomas commented Jan 4, 2019 • edited Loading

houjieth commented Jan 4, 2019

robin-thomas commented Jan 4, 2019 • edited Loading

rauljordan commented Jan 4, 2019

robin-thomas commented Jan 7, 2019 • edited Loading

rauljordan commented Jan 8, 2019

0xKiwi commented Jan 10, 2019

rauljordan commented Jan 10, 2019

rauljordan commented Jan 10, 2019

spm32 commented Jan 10, 2019

gitcoinbot commented Jan 10, 2019

luxe commented Jan 10, 2019

prestonvanloon commented Jan 10, 2019

rauljordan commented Nov 16, 2018 •

edited

Loading

gitcoinbot commented Nov 17, 2018 •

edited

Loading

jakerockland commented Nov 21, 2018 •

edited

Loading

samparsky commented Nov 27, 2018 •

edited

Loading

samparsky commented Nov 28, 2018 •

edited

Loading

prestonvanloon commented Nov 28, 2018 •

edited

Loading

samparsky commented Dec 6, 2018 •

edited

Loading

robin-thomas commented Dec 16, 2018 •

edited

Loading

robin-thomas commented Dec 18, 2018 •

edited

Loading

robin-thomas commented Jan 1, 2019 •

edited

Loading

robin-thomas commented Jan 2, 2019 •

edited

Loading

robin-thomas commented Jan 3, 2019 •

edited

Loading

robin-thomas commented Jan 3, 2019 •

edited

Loading

robin-thomas commented Jan 3, 2019 •

edited

Loading

robin-thomas commented Jan 3, 2019 •

edited

Loading

houjieth commented Jan 4, 2019 •

edited

Loading

houjieth commented Jan 4, 2019 •

edited

Loading

robin-thomas commented Jan 4, 2019 •

edited

Loading

robin-thomas commented Jan 4, 2019 •

edited

Loading

robin-thomas commented Jan 7, 2019 •

edited

Loading