Build error on Linux

[qedawkins]

I am seeing the following failure when building on Linux starting from 6d68373. The following is the critical part of the compilation error I am seeing.

/home/quinn/nodwork/onnx-mlir/src/Builder/FrontendDialectHelper.cpp:155:27: error: no matching constructor for initialization of 'ArrayRefOrSmallVector' (aka 'llvm::ArrayRef<float>')
    ArrayRefOrSmallVector array(data.begin(), data.end());
                          ^     ~~~~~~~~~~~~~~~~~~~~~~~~
/home/quinn/nodwork/onnx-mlir/src/Builder/FrontendDialectHelper.cpp:187:12: note: in instantiation of function template specialization 'onnx_mlir::createDenseElmAttr<float>' requested here
    return createDenseElmAttr<float>(externalDataDir, tp, denseBuilder);
           ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:78:18: note: candidate constructor not viable: no known conversion from 'google::protobuf::RepeatedField<float>::const_iterator' (aka 'RepeatedIterator<const float>') to 'const float *' for 1st argument
    /*implicit*/ ArrayRef(const T *data, size_t length)
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:82:5: note: candidate constructor not viable: no known conversion from 'google::protobuf::RepeatedField<float>::const_iterator' (aka 'RepeatedIterator<const float>') to 'const float *' for 1st argument
    ArrayRef(const T *begin, const T *end)
    ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:125:5: note: candidate template ignored: could not match 'ArrayRef' against 'RepeatedIterator'
    ArrayRef(const ArrayRef<U *> &A,
    ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:134:18: note: candidate template ignored: could not match 'SmallVectorTemplateCommon' against 'RepeatedIterator'
    /*implicit*/ ArrayRef(
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:143:5: note: candidate template ignored: could not match 'vector' against 'RepeatedIterator'
    ArrayRef(const std::vector<U *, A> &Vec,
    ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:71:18: note: candidate constructor not viable: requires 1 argument, but 2 were provided
    /*implicit*/ ArrayRef(NoneType) {}
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:74:18: note: candidate constructor not viable: requires single argument 'OneElt', but 2 arguments were provided
    /*implicit*/ ArrayRef(const T &OneElt)
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:89:18: note: candidate constructor template not viable: requires single argument 'Vec', but 2 arguments were provided
    /*implicit*/ ArrayRef(const SmallVectorTemplateCommon<T, U> &Vec)
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:95:18: note: candidate constructor template not viable: requires single argument 'Vec', but 2 arguments were provided
    /*implicit*/ ArrayRef(const std::vector<T, A> &Vec)
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:100:28: note: candidate constructor template not viable: requires single argument 'Arr', but 2 arguments were provided
    /*implicit*/ constexpr ArrayRef(const std::array<T, N> &Arr)
                           ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:105:28: note: candidate constructor template not viable: requires single argument 'Arr', but 2 arguments were provided
    /*implicit*/ constexpr ArrayRef(const T (&Arr)[N]) : Data(Arr), Length(N) {}
                           ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:115:18: note: candidate constructor not viable: requires single argument 'Vec', but 2 arguments were provided
    /*implicit*/ ArrayRef(const std::initializer_list<T> &Vec)
                 ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:42:40: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
  class LLVM_GSL_POINTER [[nodiscard]] ArrayRef {
                                       ^
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:42:40: note: candidate constructor (the implicit move constructor) not viable: requires 1 argument, but 2 were provided
/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:68:18: note: candidate constructor not viable: requires 0 arguments, but 2 were provided
    /*implicit*/ ArrayRef() = default;

I am on LLVM commit hash 59548fe873d8d98e359fb21fbb2a0852fed17ff5. Please let me know if there is anything else needed from me to reproduce and/or fix the error.

[sorenlassen]

thanks for reporting this

given that this code passes all the CI pipelines, I wonder what's different for you

which compiler are you using? (if you followed the linux installation instructions here it would be /usr/bin/c++ and then you can see the version with /usr/bin/c++ --version)

[qedawkins]

For /usr/bin/c++ it is GCC version 12.2. I also tried with clang version 14.0.6

[sorenlassen]

I run Apple clang version 14.0.0 on Mac, so it doesn't seem to be a compiler issue if you see this with clang v14

I can't think of any explanation

it's not a good answer but the best I can think of is to delete your build dir and try to redo the linux installation and try again

[qedawkins]

No problem, thanks for the suggestions. From what I can tell the problem is that it doesn't know how to do this conversion

/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:82:23: note:   no known conversion for argument 1 from ‘google::protobuf::RepeatedField<long unsigned int>::const_iterator’ {aka ‘google::protobuf::internal::RepeatedIterator<const long unsigned int>’} to ‘const long unsigned int*’
   82 |     ArrayRef(const T *begin, const T *end)

so it feels like either a compiler problem or maybe my protobuf version is incompatible? I was thinking of trying the docker setup recommended in the readme next, but is there a way to see the versions of everything used in the CI? That might help me with debugging this as well.

[sorenlassen]

you could try the alternative implementation here:
sorenlassen@ac7aeac

and see if that works, or generates a different compilation error which could throw some light on the issue

[gongsu832]

I can compile the latest commit df104fb fine with gcc 12 although I do get some warnings. The Dockerfiles for building llvm-project and onnx-mlir used by the Jenkins CI are docker/Dockerfile.llvm-project and docker/Dockerfile.onnx-mlir[-dev]. The typical build process involves:

• compile and install protobuf, note besides the libprotobuf libraries, you also need the python binding and their versions should match
• compile llvm-project
• compile and install onnx that comes with onnx-mlir under third_party/onnx
• compile onnx-mlir

We've seen quite a few protobuf related problems and they typically are due to one of the following:

• protobuf version is too low or too new
• libprotobuf and python binding versions mismatch
• llvm-project, onnx, and/or onnx-mlir are built against different versions of protobuf, because after updating protobuf you only rebuild one of them
• llvm-project, onnx, and/or onnx-mlir may detect different versions of python3 (so watch their cmake output) if you have multiple python versions installed
• cmake caches stuff and you should never use "make clean" when rebuilding. Instead remove everything under the build tree and start from scratch.

This and many other trickeries for setting up the build env are the reason why we recommend using the onnxmlirczar/onnx-mlir-dev docker image for development.

[AlexandreEichenberger]

If this works out for you now @qedawkins please close

[qedawkins]

Apologies I was sucked into something else for the past few days. The revision from @sorenlassen works with a small revision

 createDenseElmAttrFromProtoData(const google::protobuf::RepeatedField<U> &data,
     DenseElementsAttrBuilder<T> denseBuilder) {
-  return denseBuilder(llvm::makeArrayRef(data.begin(), data.end()));
+  return denseBuilder(llvm::makeArrayRef(data.data(), data.size()));
 }

But I haven't gone through the steps of checking with the docker image yet. It seems as though the problem is that my compiler doesn't like the conversion between the protobuf::RepeatedIterator<const int> iterator and const int*

/home/quinn/nodwork/onnx-mlir/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:487:15: note: candidate template ignored: could not match 'const T *' against 'google::protobuf::RepeatedField<int>::const_iterator' (aka 'RepeatedIterator<const int>')
  ArrayRef<T> makeArrayRef(const T *begin, const T *end) {
              ^

So there is still a problem of some sort on my end but this issue can be closed. Thanks @sorenlassen for the help!

[martin-luecke]

This is still an issue on my system as well.

The versions of the requirements on my system are:
Python 3.10.6
libprotoc 3.21.5
cmake version 3.24.1
GNU Make 4.3
Ninja 1.11.1

and I used clang version 14.0.6.

With sorenlassen@ac7aeac and the manual fixes detailed above I can compile onnx-mlir again. So would it make sense to pull them into main?

[AlexandreEichenberger]

@martin-luecke yes, please, having these fixes in main would make sense. It has to passes all of the CIs, obviously, which are dominated by Linux & Windows solutions.

[sorenlassen]

looks like this issue has resurfaced

I created a PR #1758 with my workaround from last week, please try it out

[mikeessen]

I was getting the compile error described in this issue on my mac. I tried the changes in PR #1758 but needed to make the update @qedawkins described in order to compile successfully:

• return denseBuilder(llvm::makeArrayRef(data.begin(), data.end()));

• return denseBuilder(llvm::makeArrayRef(data.data(), data.size()));

[sorenlassen]

thanks for pointing this out, I had overlooked @qedawkins's update

I updated PR #1758, please chime in on the PR if it solves your problems now (I can't reproduce the problem in my setup) and then I'll try to get it landed ASAP

nonetheless, it's frustrating that we cannot figure out why your compilers differ from the CI pipelines - do you see any explanation why? check out @gongsu832's input here:
#1729 (comment)

[gongsu832]

The problem is likely that your protobuf version 3.21.5 is too new. I updated my comments above to include this case.

[mikeessen]

@sorenlassen thanks for updating PR #1758 . I compiled with the latest updates fine.

@gongsu832 my protoc version says 3.21.6. Which is even new than 3.21.5. So as you recommend using the docker image for development would be best I think.