crypto/secp256k1: libsecp256k1/include/secp256k1.h error when using go mod command with build flag '-mod=vendor'

[sanguohot]

Hi there,

please note that this is an issue tracker reserved for bug reports and feature requests.

For general questions please use the gitter channel or the Ethereum stack exchange at https://ethereum.stackexchange.com.

System information

geth: use the latest tag=> v1.8.15

NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"

Expected behaviour

Well I test the secpk256k1 using the go mod command, because of the fucking GFW, I want to build my project with the flag -mod=vendor

Actual behaviour

It failed with vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:44: fatal error: libsecp256k1/include/secp256k1.h: No such file or directory.

Steps to reproduce the behaviour

This is my simple test code:

package main

import (
        "fmt"
        "github.com/ethereum/go-ethereum/common/hexutil"
        "github.com/ethereum/go-ethereum/crypto"
        "github.com/ethereum/go-ethereum/crypto/secp256k1"
        "log"
)

func main()  {
        bytes, err := hexutil.Decode("0x71b9732fa43b52d8c9085c053064b55dccc8785da7f12b3812787ee3def367ba0a8796dafb76fd5a146c226036092b7f5dd59feecb92776f7bb2cf2a48594ae201")
        if err != nil {
                log.Fatal(err)
        }
        pub, err := secp256k1.RecoverPubkey(crypto.Keccak256Hash([]byte("hello")).Bytes(), bytes)
        if err != nil {
                log.Fatal(err)
        }
        fmt.Println(hexutil.Encode(pub))
}

And this is my test step

> root@sanguohot:/opt# mkdir test
> root@sanguohot:/opt# cp ./medichain/test/secp256k1.test.go test/
> root@sanguohot:/opt# cd test/
> root@sanguohot:/opt/test# go mod init github.com/sanguohot/test
> go: creating new go.mod: module github.com/sanguohot/test
> root@sanguohot:/opt/test# go mod tidy && go mod vendor
> go: finding github.com/ethereum/go-ethereum/common/hexutil latest
> go: finding github.com/ethereum/go-ethereum/crypto latest
> go: finding github.com/ethereum/go-ethereum/crypto/secp256k1 latest
> go: finding github.com/ethereum/go-ethereum/common latest
> go: finding github.com/btcsuite/btcd/btcec latest
> go: finding github.com/btcsuite/btcd latest
> go: finding gopkg.in/check.v1 latest
> go: finding github.com/davecgh/go-spew/spew latest
> root@sanguohot:/opt/test# go build -mod=vendor secp256k1.test.go
> # github.com/ethereum/go-ethereum/crypto/secp256k1
> vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:44: fatal error: 
> libsecp256k1/include/secp256k1.h: No such file or directory
> compilation terminated.

I have already raised an issue golang/go#27667
Any help would be greatly appreciated.

[andyvauliln]

How is going on? I have the same issue
image: ethereum/client-go:latest
entrypoint: dlv debug server.go -l 0.0.0.0:2345 --headless=true --log=true -- server

[sanguohot]

@andyvauliln #26366 (comment)
works to me.

[adamschmideg]

Closing this issue as solved for the reporter. @andyvauliln if you still experience a problem, please report an issue.

[gracenoah]

The ideal solution to this problem would be to have a native go version of the library so that users of go-ethereum as a library would have the option to not use cgo and have their dependencies just work.

There is a pure go version at https://github.com/btcsuite/btcd/tree/master/btcec

The geth binary can continue to be compiled with the c version, but having the option to just use Go would simplify things significantly for many of us.

My actual problem is that we want to be able to use https://renovatebot.com to automatically upgrade our dependencies but it doesn't have any concept of post-upgrade commands to run, so it can't automatically upgrade anything because after running go mod vendor, the libsecp256k1/src directory is deleted.

A second idea for improving the situation: all .c and .h files could be moved directly into the go package directory so that they are not dropped by go mod vendor.

[ryanleecode]

I am also having this problem.

# github.com/ethereum/go-ethereum/crypto/secp256k1
vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:10: fatal error: libsecp256k1/include/secp256k1.h: No such file or directory
 #include "libsecp256k1/include/secp256k1.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

[shiruitao]

I am also having this problem.

# github.com/ethereum/go-ethereum/crypto/secp256k1
vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:10: fatal error: libsecp256k1/include/secp256k1.h: No such file or directory
 #include "libsecp256k1/include/secp256k1.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

export GOFLAGS=""

[skliarovartem]

have same problem in Goland

# github.com/ethereum/go-ethereum/crypto/secp256k1
vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:10: fatal error: libsecp256k1/include/secp256k1.h file not found
#include "libsecp256k1/include/secp256k1.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
# github.com/karalabe/usb
vendor/github.com/karalabe/usb/hid_enabled.go:22:10: fatal error: ./hidapi/hidapi/hidapi.h file not found
#include "./hidapi/hidapi/hidapi.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Compilation finished with exit code 2

[shikingram]

I am also having this problem. how to fix it ?

my go-ethereum version: v1.9.21

# github.com/ethereum/go-ethereum/crypto/secp256k1
../../../../vendor/github.com/ethereum/go-ethereum/crypto/secp256k1/curve.go:42:10: fatal error: 'libsecp256k1/include/secp256k1.h' file not found
#include "libsecp256k1/include/secp256k1.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

[alboto]

The ideal solution to this problem would be to have a native go version of the library so that users of go-ethereum as a library would have the option to not use cgo and have their dependencies just work.

There is a pure go version at https://github.com/btcsuite/btcd/tree/master/btcec

The geth binary can continue to be compiled with the c version, but having the option to just use Go would simplify things significantly for many of us.

My actual problem is that we want to be able to use https://renovatebot.com to automatically upgrade our dependencies but it doesn't have any concept of post-upgrade commands to run, so it can't automatically upgrade anything because after running go mod vendor, the libsecp256k1/src directory is deleted.

A second idea for improving the situation: all .c and .h files could be moved directly into the go package directory so that they are not dropped by go mod vendor.

Cuz I just want to use secp256k1 signature, so it works for me!
just use the following package
https://pkg.go.dev/github.com/btcsuite/btcd/btcec/v2

[kabaluyot]

Frequently encountered this issue on mac.

This issue was described here: https://blog.csdn.net/ic_xcc/article/details/124456791

You need to manually copy this https://github.com/ethereum/go-ethereum/tree/1816cdc9fdad7af6085369efb9864d934aae2c81/crypto/secp256k1/libsecp256k1 folder to your project's vendor/github.com/ethereum/go-ethereum/crypto/secp256k1 folder. Notice that the libsecp256k1 is missing in your vendor go-ethereum version

I downloaded the libsecp256k1 directly here from github and copy pasted it there (rename folder to libsecp256k1): https://download-directory.github.io/?url=https%3A%2F%2Fgithub.com%2Fethereum%2Fgo-ethereum%2Ftree%2F1816cdc9fdad7af6085369efb9864d934aae2c81%2Fcrypto%2Fsecp256k1%2Flibsecp256k1