[R] Install fails with LTO flags #28586

asfimport · 2021-05-22T20:51:25Z

Hi,

I noticed my installation failed after updating R to the newly released 4.1.0. Arrow compiles without error, but installation fails with a segfault when R tests whether the package can be loaded.
Here are the relevant environment variables I've set:
LIBARROW_MINIMAL: false
LIBARROW_BINARY: ubuntu-20.04 (also fails with false)
ARROW_R_DEV: true
Let me know if there are other configurations you'd like me to test.

Related: ARROW-12824

The install log:


install.packages("arrow")
Installing package into ‘/home/karl/.R/4.1’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/arrow_4.0.0.1.tar.gz'
Content type 'application/x-gzip' length 426715 bytes (416 KB)
==================================================
downloaded 416 KB

* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** Using ubuntu-18.04 binary for ubuntu-20.04
trying URL 'https://arrow-r-nightly.s3.amazonaws.com/libarrow/bin/ubuntu-18.04/arrow-4.0.0.1.zip'
Content type 'binary/octet-stream' length 16007220 bytes (15.3 MB)
==================================================
downloaded 15.3 MB

*** Successfully retrieved C++ binaries for ubuntu-18.04
**** Binary package requires libcurl and openssl
**** If installation fails, retry after installing those system requirements
PKG_CFLAGS=-I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3
PKG_LIBS=-L/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/lib -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto -lcurl
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c array.cpp -o array.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c array_to_vector.cpp -o array_to_vector.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c arraydata.cpp -o arraydata.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c arrowExports.cpp -o arrowExports.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c buffer.cpp -o buffer.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c chunkedarray.cpp -o chunkedarray.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c compression.cpp -o compression.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c compute.cpp -o compute.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c csv.cpp -o csv.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c dataset.cpp -o dataset.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c datatype.cpp -o datatype.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c expression.cpp -o expression.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c feather.cpp -o feather.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c field.cpp -o field.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c filesystem.cpp -o filesystem.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c imports.cpp -o imports.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c io.cpp -o io.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c json.cpp -o json.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c memorypool.cpp -o memorypool.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c message.cpp -o message.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c parquet.cpp -o parquet.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c py-to-r.cpp -o py-to-r.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c r_to_arrow.cpp -o r_to_arrow.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c recordbatch.cpp -o recordbatch.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c recordbatchreader.cpp -o recordbatchreader.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c recordbatchwriter.cpp -o recordbatchwriter.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c runtimeinfo.cpp -o runtimeinfo.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c scalar.cpp -o scalar.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c schema.cpp -o schema.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c symbols.cpp -o symbols.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c table.cpp -o table.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c threadpool.cpp -o threadpool.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/include  -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -DARROW_R_WITH_S3 -I'/home/karl/.R/4.1/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c type_infer.cpp -o type_infer.o
g++ -std=gnu++11 -Wl,--strip-debug -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro -o arrow.so array.o array_to_vector.o arraydata.o arrowExports.o buffer.o chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o expression.o feather.o field.o filesystem.o imports.o io.o json.o memorypool.o message.o parquet.o py-to-r.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o runtimeinfo.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/tmp/RtmpGwVvlP/R.INSTALL106e64cdb5ef7/arrow/libarrow/arrow-4.0.0.1/lib -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto -lcurl -L/usr/lib/R/lib -lR
installing to /home/karl/.R/4.1/00LOCK-arrow/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location

 *** caught segfault ***
address (nil), cause 'memory not mapped'

Traceback:
 1: dyn.load(file, DLLpath = DLLpath, ...)
 2: library.dynam(lib, package, package.lib)
 3: loadNamespace(package, lib.loc)
 4: doTryCatch(return(expr), name, parentenv, handler)
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6: tryCatchList(expr, classes, parentenv, handlers)
 7: tryCatch({    attr(package, "LibPath") <- which.lib.loc    ns <- loadNamespace(package, lib.loc)    env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) {    P <- if (!is.null(cc <- conditionCall(e)))         paste(" in", deparse(cc)[1L])    else ""    msg <- gettextf("package or namespace load failed for %s%s:\n %s",         sQuote(package), P, conditionMessage(e))    if (logical.return)         message(paste("Error:", msg), domain = NA)    else stop(msg, call. = FALSE, domain = NA)})
 8: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)
 9: withCallingHandlers(expr, packageStartupMessage = function(c) tryInvokeRestart("muffleMessage"))
10: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE))
11: doTryCatch(return(expr), name, parentenv, handler)
12: tryCatchOne(expr, names, parentenv, handlers[[1L]])
13: tryCatchList(expr, classes, parentenv, handlers)
14: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
15: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib,     character.only = TRUE, logical.return = TRUE)))
16: tools:::.test_load_package("arrow", "/home/karl/.R/4.1/00LOCK-arrow/00new")
An irrecoverable exception occurred. R is aborting now ...
Segmentation fault (core dumped)
ERROR: loading failed
* removing ‘/home/karl/.R/4.1/arrow’

Environment: R 4.1.0
Reporter: Karl Dunkle Werner / @karldw
Assignee: Jonathan Keane / @jonkeane

Related issues:

[R] Unable to install on Ubuntu 21.04 (is duplicated by)

Original Issue Attachments:

Externally tracked issue: #10611

_{Note: This issue was originally created as ARROW-12853. Please see the migration documentation for further details.}

The text was updated successfully, but these errors were encountered:

asfimport · 2021-05-24T18:07:01Z

Neal Richardson / @nealrichardson:
Thanks for the report. Do you still have R 4.0 installed? Does which R point to the new R? I'm wondering if there's a mismatch in compiler (flags) somehow after you upgraded R.

Could you try with LIBARROW_BINARY=false ARROW_R_DEV=true and attach the full logs? That might tell us more.

asfimport · 2021-05-24T23:45:02Z

Karl Dunkle Werner / @karldw:
No problem! I don't have R 4.0 installed anymore. which R points to /usr/bin/R and R --version gives 4.1.0.

With a bit more testing, I see that I'm able to install arrow in docker (using rocker v4.1.0), but still can't install on my computer. The generated LDFLAGS are different between the two: in docker, I get "-Wl,-z,relro", while outside docker I get "-Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro". (These flags remain different even after I move my Makevars file somewhere else.) Both logs are attached.

asfimport · 2021-05-25T17:17:25Z

Neal Richardson / @nealrichardson:
Wild guess is that LTO is the issue (-flto=auto). Can you edit R's Makevars to remove that?

asfimport · 2021-05-25T18:28:06Z

Karl Dunkle Werner / @karldw:
Interesting. So I can prevent that flag by using:

withr::with_makevars(list(LDFLAGS = ""), install.packages("arrow"))

However, when I do that, the build still fails. I notice that there are still LTO-related flags being passed to g++, for instance:

g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/tmp/Rtmp9mp2UD/R.INSTALL316d2ebe2d9b/arrow/libarrow/arrow-4.0.0.1/include -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -I'/home/karl/.R/x86_64-pc-linux-gnu-library/4.1/cpp11/include' -fpic -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c array.cpp -o array.o

My understanding of the Makevars file is that my user file in $HOME/.R/Makevars is not used because the arrow build provides its own. However, I think the /etc/R/Makeconf is still used (details). I noticed the Makeconf file differs between my computer and docker. My computer adds extra LTO flags, though I'm pretty sure I've never edited it. Both are attached.

asfimport · 2021-05-25T21:10:21Z

Neal Richardson / @nealrichardson:
When compiling the C++ library, arrow uses the flags that R provides: https://github.com/apache/arrow/blob/master/r/tools/nixlibs.R#L325-L329 This is so that the C++ library and the R bindings are compiled with the same compiler and flags. Arrow does not provide or override Makevars.

Based on what you're saying, I wonder if LTO flags are coming from somewhere else in the R config, and since they're not being passed to the Arrow C++ build, we have a mismatch (also, I'm not sure we can just find them and pass them to the C++ build because I seem to recall from trying before that Arrow did not build with LTO enabled).

asfimport · 2021-05-25T23:32:00Z

Karl Dunkle Werner / @karldw:
I see, thanks!
In that Makeconf file from my host machine (Makeconf_local), the LTO flags are in the fields CFLAGS, CXXFLAGS, CXX11FLAGS, etc. These show up in my R config:

tools::Rcmd("config CXX11FLAGS")
#> -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g

When I copy the Makeconf file from my host machine, Makeconf_local, into the docker container, Arrow fails to install with the same error as on my host machine. I think these steps should be reproducible:

Download Makeconf_local from this issue.
Start docker: docker run -e "LIBARROW_BINARY=false" -e "ARROW_R_DEV=true" -e "LIBARROW_MINIMAL=true" --rm -ti rocker/r-base:4.1.0
In a different terminal, find the docker container ID: docker ps
Copy Makeconf: docker cp Makeconf_local <container ID>:/etc/R/Makeconf
In the docker R:
- tools::Rcmd("config CXX11FLAGS") to see the new flags
- install.packages("arrow") now fails
  
  I'm guessing my configuration came from this deb file: https://cloud.r-project.org/bin/linux/ubuntu/hirsute-cran40/r-base-core_4.1.0-1.2104.0_amd64.deb, where, after unzipping, the file data/etc/R/Makeconf is the same as the attached Makeconf_local.
  
  Additionally, the issue isn't specific to R 4.1.0. Installing Arrow fails with this Makeconf file when I do the same steps above using the rocker/r-base:4.0.5 image.

asfimport · 2021-05-26T13:57:23Z

Neal Richardson / @nealrichardson:
Thanks for tracking that down. CXX11FLAGS are one set of flags we don't pass to the Arrow C++ build because (IIRC) it can collide/override the Arrow C++ build's optimization/debug level. But in this case there seem to be other flags in there that, if they're not passed, cause a mismatch.

You could try setting the env var EXTRA_CMAKE_FLAGS='-DARROW_CXXFLAGS="-flto=auto -ffat-lto-objects"', that will pass those flags to the Arrow C++ build. It would be interesting to see if Arrow can build with that; our previous explorations of LTO were not successful (see ARROW-9616).

asfimport · 2021-05-27T16:26:38Z

Karl Dunkle Werner / @karldw:
With that environment variable set, the relevant parts of the log become:


+ /usr/bin/cmake -DARROW_BOOST_USE_SHARED=OFF -DARROW_BUILD_TESTS=OFF -DARROW_BUILD_SHARED=OFF -DARROW_BUILD_STATIC=ON -DARROW_COMPUTE=ON -DARROW_CSV=ON -DARROW_DATASET=ON -DARROW_DEPENDENCY_SOURCE=BUNDLED -DARROW_FILESYSTEM=ON -DARROW_JEMALLOC=OFF -DARROW_MIMALLOC=OFF -DARROW_JSON=ON -DARROW_PARQUET=ON -DARROW_S3=OFF -DARROW_WITH_BROTLI=OFF -DARROW_WITH_BZ2=OFF -DARROW_WITH_LZ4=OFF -DARROW_WITH_RE2=ON -DARROW_WITH_SNAPPY=OFF -DARROW_WITH_UTF8PROC=ON -DARROW_WITH_ZLIB=OFF -DARROW_WITH_ZSTD=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_PREFIX=/tmp/RtmpvaRJ9j/R.INSTALL3dfc55511a7e/arrow/libarrow/arrow-4.0.0.1 -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON -DCMAKE_UNITY_BUILD=ON '-DARROW_CXXFLAGS="-flto=auto' '-ffat-lto-objects"' -G 'Unix Makefiles' /tmp/RtmphF0pgX/file3e221c541529/apache-arrow-4.0.0/cpp
...

-- CMAKE_C_FLAGS:  -O3 -DNDEBUG  -Wall -fno-semantic-interposition -msse4.2 "-flto=auto
-- CMAKE_CXX_FLAGS:  -Wno-noexcept-type -Wno-subobject-linkage  -fdiagnostics-color=always -O3 -DNDEBUG  -Wall -fno-semantic-interposition -msse4.2 "-flto=auto
...

Scanning dependencies of target arrow_dataset_objlib
[ 44%] Building CXX object src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o
/bin/sh: 1: Syntax error: Unterminated quoted string
make[2]: *** [src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/build.make:82: src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/Unity/unity_1_cxx.cxx.o] Error 2
make[1]: *** [CMakeFiles/Makefile2:1450: src/arrow/dataset/CMakeFiles/arrow_dataset_objlib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

So it looks like the quoted string isn't getting parsed properly. (See "-flto=auto.)

If I instead do EXTRA_CMAKE_FLAGS='-DARROW_CXXFLAGS=-flto=auto -ffat-lto-objects', the Arrow build only picks up the -flto=auto flag, and the R compilation generates a bunch of -Wodr warnings before failing.
If I do EXTRA_CMAKE_FLAGS='-DARROW_CXXFLAGS=-flto=auto -DARROW_CXXFLAGS=-ffat-lto-objects', only the -ffat-lto-objects flag is included. Both of those still end in a segfault.

Is there a better way to pass multiple flags?


g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro -o arrow.so array.o array_to_vector.o arraydata.o arrowExports.o buffer.o chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o expression.o feather.o field.o filesystem.o imports.o io.o json.o memorypool.o message.o parquet.o py-to-r.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o runtimeinfo.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/tmp/RtmpKRWweq/R.INSTALL2b7b181077e9/arrow/libarrow/arrow-4.0.0.1/lib -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -L/usr/lib/R/lib -lR
/tmp/RtmpKRWweq/R.INSTALL2b7b181077e9/arrow/libarrow/arrow-4.0.0.1/include/arrow/io/interfaces.h:305:20: warning: virtual table of type ‘struct WritableFile’ violates one definition rule [-Wodr]
  305 | class ARROW_EXPORT WritableFile : public OutputStream, public Seekable {
      |                    ^
/tmp/RtmpsSjESj/file2ba1153cf93c/apache-arrow-4.0.0/cpp/src/arrow/io/interfaces.h:305: note: the conflicting type defined in another translation unit
<built-in>: note: virtual method ‘__cxa_pure_virtual’
/tmp/RtmpKRWweq/R.INSTALL2b7b181077e9/arrow/libarrow/arrow-4.0.0.1/include/arrow/io/interfaces.h:124:18: note: ought to match virtual method ‘Abort’ but does not
  124 |   virtual Status Abort();
      |                  ^
/tmp/RtmpsSjESj/file2ba1153cf93c/apache-arrow-4.0.0/cpp/src/arrow/io/slow.h:51: warning: virtual table of type ‘struct SlowInputStreamBase’ violates one definition rule [-Wodr]
/tmp/RtmpsSjESj/file2ba1153cf93c/apache-arrow-4.0.0/cpp/src/arrow/io/slow.h:51: note: the conflicting type defined in another translation unit
/tmp/RtmpsSjESj/file2ba1153cf93c/apache-arrow-4.0.0/cpp/src/arrow/io/interfaces.cc:91: note: virtual method ‘io_context’
<built-in>: note: ought to match virtual method ‘__cxa_pure_virtual’ but does not
/tmp/RtmpsSjESj/file2ba1153cf93c/apache-arrow-4.0.0/cpp/src/arrow/io/concurrency.h:162: warning: virtual table of type ‘struct RandomAccessFileConcurrencyWrapper’ violates one definition rule [-Wodr]
/tmp/RtmpKRWweq/R.INSTALL2b7b181077e9/arrow/libarrow/arrow-4.0.0.1/include/arrow/io/concurrency.h:162:20: note: the conflicting type defined in another translation unit
  162 | class ARROW_EXPORT RandomAccessFileConcurrencyWrapper : public RandomAccessFile {
      |                    ^
/tmp/RtmpsSjESj/file2ba1153cf93c/apache-arrow-4.0.0/cpp/src/arrow/io/interfaces.cc:91: note: virtual method ‘io_context’
<built-in>: note: ought to match virtual method ‘__cxa_pure_virtual’ but does not

** testing if installed package can be loaded from temporary location

 *** caught segfault ***
address (nil), cause 'memory not mapped'

In ARROW-9616, you said "If CRAN decides that it LTO's everything, then we wouldn't be able to turn that off." The release notes for R 4.1.0 say "Packages can opt in or out of LTO compilation via a UseLTO field in the ‘DESCRIPTION’ file. (As usual this can be overridden by the command-line flags.)" So it sounds like they're not going to force LTO, at least for now.

asfimport · 2021-05-27T18:30:19Z

Neal Richardson / @nealrichardson:
Hmm, what if you added UseLTO: FALSE to the DESCRIPTION?

asfimport · 2021-05-27T22:18:22Z

Karl Dunkle Werner / @karldw:
It still segfaults in the same place. Maybe the flags are overriding the UseLTO option?

asfimport · 2021-05-27T22:23:36Z

Neal Richardson / @nealrichardson:
Yeah IDK, this is new territory. As for the previous issue about quoting in the env var, I'm sure there's a solution but I don't know it off the top of my head.

asfimport · 2021-06-02T02:33:30Z

Karl Dunkle Werner / @karldw:
Would it be reasonable to pass -fno-lto, maybe in the LDFLAGS? Even if the files are compiled with -flto, they can still be linked without LTO (as long as -fno-fat-lto-objects isn't enabled).

asfimport · 2021-06-07T15:20:22Z

Neal Richardson / @nealrichardson:
Did you get a chance to try this?

asfimport · 2021-06-07T21:25:34Z

Karl Dunkle Werner / @karldw:
It seems like LDFLAGS doesn't get passed to the R package compilation in the way I assumed it would.

When I edit my user $HOME/.R/Makevars, adding the line LDFLAGS+=-fno-lto, the flag is passed to both the Arrow library compilation and the R linking step. Everything works!

However, I tried to add this flag to the build file and couldn't get them to work.

Editing nixlibs.R to set LDFLAGS = paste(R_CMD_config("LDFLAGS"), "-fno-lto") passes the flag for the Arrow library compilation, but not for the R package.
Edits to Makevars.in to add LDFLAGS+=-fno-lto show up in the generated Makevars file, but the flag isn't used in the R package compilation.
Same if I instead write LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro -fno-lto
Setting CXX11FLAGS="${CXX11FLAGS} -fno-lto" in the configure shell script also isn't picked up. (Which makes sense, as the modification is only used for the test header.)
Setting the environment variable ARROW_R_CXXFLAGS to -fno-lto is picked up for R package compilation of individual files, but not for linking.

It's very possible I'm misunderstanding how LDFLAGS are handled. Do you have any tips?

I also had a bit of a conversation about LTO flags over on R-SIG-Debian.

The result there was to do one of the following:

Remove the flags -flto=auto and -ffat-lto-objects from the CXX11FLAGS in the system /etc/R/Makeconf
Set CXX11FLAGS in your user Makevars (in $HOME/.R/Makevars) to something
like this:
```
CXX11FLAGS = -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g
```
(This is just the default Ubuntu 21.04 R 4.1.0 Makeconf CXX11FLAGS with LTO flags removed).

A third option, as I mentioned above, would be LDFLAGS+=-fno-lto in the user Makevars.

asfimport · 2021-06-07T22:04:28Z

Neal Richardson / @nealrichardson:
Thanks for digging in. IIUC the issue is that the Arrow C++ library and the R bindings need to be compiled the same way, or at least compatibly. You are right that nixlibs.R only affects the C++ library compilation, and ARROW_R_CXXFLAGS only affects the R bindings. Could you try adding to both at the same time?

asfimport · 2021-06-07T22:18:31Z

Karl Dunkle Werner / @karldw:
Unfortunately, it seems like the ARROW_R_CXXFLAGS does get used when the R object files are compiled, but doesn't get used during linking. Loading the package fails with the same segfault, I think because -flto is still passed and there's no overriding -fno-lto.

The last few lines of the R package compilation process are:


g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I/home/karl/arrow/r/libarrow/arrow-4.0.0.9000/include  -fno-lto -DARROW_R_WITH_ARROW -DARROW_R_WITH_PARQUET -DARROW_R_WITH_DATASET -I'/home/karl/test_r_lib/cpp11/include'    -fpic  -g -O2 -ffile-prefix-map=/build/r-base-aXXzqd/r-base-4.1.0=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c type_infer.cpp -o type_infer.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro -o arrow.so array.o array_to_vector.o arraydata.o arrowExports.o buffer.o chunkedarray.o compression.o compute.o config.o csv.o dataset.o datatype.o expression.o feather.o field.o filesystem.o imports.o io.o json.o memorypool.o message.o parquet.o py-to-r.o r_to_arrow.o recordbatch.o recordbatchreader.o recordbatchwriter.o scalar.o schema.o symbols.o table.o threadpool.o type_infer.o -L/home/karl/arrow/r/libarrow/arrow-4.0.0.9000/lib -larrow_dataset -lparquet -larrow -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -L/usr/lib/R/lib -lR
installing to /home/karl/test_r_lib/00LOCK-r/00new/arrow/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location

 *** caught segfault ***
address (nil), cause 'memory not mapped'

asfimport · 2021-06-07T22:20:44Z

Neal Richardson / @nealrichardson:
Ah right, it's a link-time issue.

asfimport · 2021-06-09T17:48:01Z

Karl Dunkle Werner / @karldw:
One combination that does work is to add -fno-lto to nixlibs.R and also change the Makevars.in line:
-PKG_LIBS=@libs@
+PKG_LIBS=@libs@ -fno-lto

So that the -fno-lto flag gets passed at link-time. I'm happy to send a PR, but this isn't exactly the intended use of PKG_LIBS, so let me know if you don't want to use it. If you do want to use it, should it be unix-only or windows as well?

Other flags that are more appropriate, like LDFLAGS, get overridden (I think; I'm not positive in the following docs quote whether the system Makeconf file is also read).

If the src subdirectory of an add-on package contains source code with one of the extensions listed above or a file Makevars but not a file Makefile, R CMD INSTALL creates a shared object (for loading into R through useDynlib in the NAMESPACE, or in the .onLoad function of the package) using the R CMD SHLIB mechanism. If file Makevars exists it is read first, then the system makefile and then any personal Makevars files.
(Emphasis added)

asfimport · 2021-06-09T18:54:29Z

Neal Richardson / @nealrichardson:
A PR would be great–especially if you have or know of a docker image that reproduces the issue so we can add that to our continuous integration.

asfimport · 2021-06-10T18:15:09Z

Karl Dunkle Werner / @karldw:
I think most environments that set -flto in the system Makeconf or user Makevars will run into this issue. One way to create such an environment would be to make a new image with a Dockerfile like this:


FROM rhub/debian-gcc-release
# Also works with the smaller rocker/r-base:4.1.0 etc

ENV LIBARROW_BINARY=false \
    LIBARROW_MINIMAL=true \
    LIBARROW_BUILD=true \
    LIBARROW_DOWNLOAD=true \
    ARROW_R_DEV=true

# Add the Makeconf flags from Ubuntu 21.04's R 4.1.0 build:
# https://cloud.r-project.org/bin/linux/ubuntu/hirsute-cran40/r-base-core_4.1.0-1.2104.0_amd64.deb
# See https://issues.apache.org/jira/browse/ARROW-12853
ADD https://issues.apache.org/jira/secure/attachment/13025950/Makeconf_local /etc/R/Makeconf

CMD Rscript -e "install.packages('arrow')"

But an easier way might be to do one of the following:
(a) Add a service to docker-compose.yml that's exactly the same as the current r service, but sets -flto before running r_test.sh.
(b) Just edit the existing r_docker_configure.sh to add -flto and stop testing R environments that don't set -flto.

Do either of these seem reasonable?

asfimport · 2021-07-02T05:29:31Z

Karl Dunkle Werner / @karldw:
I've been slow here, but these flag changes were included by [~jonkean] in ARROW-13199 and #10611. That pull request adds an Ubuntu 21.04 dockerfile that reproduces this issue, so feel free to close this when the PR is merged.

asfimport · 2021-07-06T13:31:31Z

Jonathan Keane / @jonkeane:
Resolved by ARROW-13199

asfimport closed this as completed Jul 6, 2021

asfimport assigned jonkeane Jan 10, 2023

asfimport mentioned this issue Jan 11, 2023

[R] Unable to install on Ubuntu 21.04 #28937

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[R] Install fails with LTO flags #28586

[R] Install fails with LTO flags #28586

asfimport commented May 22, 2021 •

edited

Loading

asfimport commented May 24, 2021

asfimport commented May 24, 2021

asfimport commented May 25, 2021

asfimport commented May 25, 2021

asfimport commented May 25, 2021

asfimport commented May 25, 2021

asfimport commented May 26, 2021

asfimport commented May 27, 2021

asfimport commented May 27, 2021

asfimport commented May 27, 2021

asfimport commented May 27, 2021

asfimport commented Jun 2, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 9, 2021

asfimport commented Jun 9, 2021

asfimport commented Jun 10, 2021

asfimport commented Jul 2, 2021

asfimport commented Jul 6, 2021

[R] Install fails with LTO flags #28586

[R] Install fails with LTO flags #28586

Comments

asfimport commented May 22, 2021 • edited Loading

Related issues:

Original Issue Attachments:

Externally tracked issue: #10611

asfimport commented May 24, 2021

asfimport commented May 24, 2021

asfimport commented May 25, 2021

asfimport commented May 25, 2021

asfimport commented May 25, 2021

asfimport commented May 25, 2021

asfimport commented May 26, 2021

asfimport commented May 27, 2021

asfimport commented May 27, 2021

asfimport commented May 27, 2021

asfimport commented May 27, 2021

asfimport commented Jun 2, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 7, 2021

asfimport commented Jun 9, 2021

asfimport commented Jun 9, 2021

asfimport commented Jun 10, 2021

asfimport commented Jul 2, 2021

asfimport commented Jul 6, 2021

asfimport commented May 22, 2021 •

edited

Loading