Support Freebsd

[sjolicoeur]

I would be great to be able to run vector in FreeBSD ( or any other *BSD ). So I tried to do the manual steps and encountered a few errors. The first one was solved by using gnu-make (gmake) instead of bsd-make. I am not familiar with Rust, but I hope that this can help bring vector to FreeBSD. (it shouldn't be too far from MacOS support, I hope).

Platform: FreeBSD 11.2-RELEASE-p10
cargo 1.35.0
vector Source Version: 0.4.0-dev
Env options: RUST_BACKTRACE=1

error:

 Compiling jemalloc-sys v0.3.0
error: failed to run custom build command for `leveldb-sys v2.0.1`
process didn't exit successfully: `/tmp/vector/vector-master/target/debug/build/leveldb-sys-cd5040109e6341dc/build-script-build` (exit code: 101)
--- stdout
[build] Started
[snappy] Cleaning
test -z "libsnappy.la" || rm -f libsnappy.la
rm -f "./so_locations"
rm -rf .libs _libs
 rm -f snappy_unittest
rm -f *.o
rm -f *.lo
rm -f *.tab.c
test -z "snappy-stubs-public.h" || rm -f snappy-stubs-public.h
test . = "." || test -z "" || rm -f
rm -f config.h stamp-h1
rm -f libtool config.lt
rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags
rm -f config.status config.cache config.log  configure.lineno config.status.lineno
rm -rf ./.deps
rm -f Makefile
[snappy] Configuring
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... nawk
checking whether make sets $(MAKE)... yes
checking build system type... x86_64-unknown-freebsd11.2
checking host system type... x86_64-unknown-freebsd11.2
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... no
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking dependency style of cc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by cc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm
checking the name lister (/usr/bin/nm) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... no
checking how to convert x86_64-unknown-freebsd11.2 file names to x86_64-unknown-freebsd11.2 format... func_convert_file_noop
checking how to convert x86_64-unknown-freebsd11.2 file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm output from cc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... cc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if cc supports -fno-rtti -fno-exceptions... yes
checking for cc option to produce PIC... -fPIC -DPIC
checking if cc PIC flag -fPIC -DPIC works... yes
checking if cc static flag -static works... yes
checking if cc supports -c -o file.o... yes
checking if cc supports -c -o file.o... (cached) yes
checking whether the cc linker (/usr/bin/ld) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... freebsd11.2 ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... no
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for g++... no
checking for c++... c++
checking whether we are using the GNU C++ compiler... yes
checking whether c++ accepts -g... yes
checking dependency style of c++... gcc3
checking how to run the C++ preprocessor... c++ -E
checking for ld used by c++... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking whether the c++ linker (/usr/bin/ld) supports shared libraries... yes
checking for c++ option to produce PIC... -fPIC -DPIC
checking if c++ PIC flag -fPIC -DPIC works... yes
checking if c++ static flag -static works... yes
checking if c++ supports -c -o file.o... yes
checking if c++ supports -c -o file.o... (cached) yes
checking whether the c++ linker (/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... freebsd11.2 ld.so
checking how to hardcode library paths into programs... immediate
checking whether byte ordering is bigendian... no
checking for size_t... yes
checking for ssize_t... yes
checking for stdint.h... (cached) yes
checking stddef.h usability... yes
checking stddef.h presence... yes
checking for stddef.h... yes
checking sys/mman.h usability... yes
checking sys/mman.h presence... yes
checking for sys/mman.h... yes
checking sys/resource.h usability... yes
checking sys/resource.h presence... yes
checking for sys/resource.h... yes
checking windows.h usability... no
checking windows.h presence... no
checking for windows.h... no
checking byteswap.h usability... no
checking byteswap.h presence... no
checking for byteswap.h... no
checking sys/byteswap.h usability... no
checking sys/byteswap.h presence... no
checking for sys/byteswap.h... no
checking sys/endian.h usability... yes
checking sys/endian.h presence... yes
checking for sys/endian.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for mmap... yes
checking for 'gtest-config'... checking for gtest-config... no
no
checking for pkg-config... no
checking for gflags... no
checking if the compiler supports __builtin_expect... yes
checking if the compiler supports __builtin_ctzll... yes
checking for zlibVersion in -lz... yes
checking for lzo1x_1_15_compress in -llzo2... no
checking for lzf_compress in -llzf... no
checking for fastlz_compress in -lfastlz... no
checking for qlz_compress in -lquicklz... no
configure: creating ./config.status
config.status: creating Makefile
config.status: creating snappy-stubs-public.h
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
[snappy] Building
make  all-am
/bin/sh ./libtool --tag=CXX    --mode=compile c++ -DHAVE_CONFIG_H -I.      -fPIC -MT snappy.lo -MD -MP -MF .deps/snappy.Tpo -c -o snappy.lo snappy.cc
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy.lo -MD -MP -MF .deps/snappy.Tpo -c snappy.cc  -fPIC -DPIC -o .libs/snappy.o
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy.lo -MD -MP -MF .deps/snappy.Tpo -c snappy.cc -o snappy.o >/dev/null 2>&1
mv -f .deps/snappy.Tpo .deps/snappy.Plo
/bin/sh ./libtool --tag=CXX    --mode=compile c++ -DHAVE_CONFIG_H -I.      -fPIC -MT snappy-sinksource.lo -MD -MP -MF .deps/snappy-sinksource.Tpo -c -o snappy-sinksource.lo snappy-sinksource.cc
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy-sinksource.lo -MD -MP -MF .deps/snappy-sinksource.Tpo -c snappy-sinksource.cc  -fPIC -DPIC -o .libs/snappy-sinksource.o
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy-sinksource.lo -MD -MP -MF .deps/snappy-sinksource.Tpo -c snappy-sinksource.cc -o snappy-sinksource.o >/dev/null 2>&1
mv -f .deps/snappy-sinksource.Tpo .deps/snappy-sinksource.Plo
/bin/sh ./libtool --tag=CXX    --mode=compile c++ -DHAVE_CONFIG_H -I.      -fPIC -MT snappy-stubs-internal.lo -MD -MP -MF .deps/snappy-stubs-internal.Tpo -c -o snappy-stubs-internal.lo snappy-stubs-internal.cc
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy-stubs-internal.lo -MD -MP -MF .deps/snappy-stubs-internal.Tpo -c snappy-stubs-internal.cc  -fPIC -DPIC -o .libs/snappy-stubs-internal.o
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy-stubs-internal.lo -MD -MP -MF .deps/snappy-stubs-internal.Tpo -c snappy-stubs-internal.cc -o snappy-stubs-internal.o >/dev/null 2>&1
mv -f .deps/snappy-stubs-internal.Tpo .deps/snappy-stubs-internal.Plo
/bin/sh ./libtool --tag=CXX    --mode=compile c++ -DHAVE_CONFIG_H -I.      -fPIC -MT snappy-c.lo -MD -MP -MF .deps/snappy-c.Tpo -c -o snappy-c.lo snappy-c.cc
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy-c.lo -MD -MP -MF .deps/snappy-c.Tpo -c snappy-c.cc  -fPIC -DPIC -o .libs/snappy-c.o
libtool: compile:  c++ -DHAVE_CONFIG_H -I. -fPIC -MT snappy-c.lo -MD -MP -MF .deps/snappy-c.Tpo -c snappy-c.cc -o snappy-c.o >/dev/null 2>&1
mv -f .deps/snappy-c.Tpo .deps/snappy-c.Plo
/bin/sh ./libtool --tag=CXX    --mode=link c++   -fPIC -version-info 3:1:2  -o libsnappy.la -rpath /usr/local/lib snappy.lo snappy-sinksource.lo  snappy-stubs-internal.lo snappy-c.lo
libtool: link: c++  -fPIC -DPIC -shared -nostdlib /usr/lib/crti.o /usr/lib/crtbeginS.o  .libs/snappy.o .libs/snappy-sinksource.o .libs/snappy-stubs-internal.o .libs/snappy-c.o   -L/usr/lib -lc++ -lm -lc -lgcc -lgcc_s /usr/lib/crtendS.o /usr/lib/crtn.o    -Wl,-soname -Wl,libsnappy.so.3 -o .libs/libsnappy.so.3
libtool: link: (cd ".libs" && rm -f "libsnappy.so" && ln -s "libsnappy.so.3" "libsnappy.so")
libtool: link: (cd ".libs" && rm -f "libsnappy.so" && ln -s "libsnappy.so.3" "libsnappy.so")
libtool: link: ar cru .libs/libsnappy.a  snappy.o snappy-sinksource.o snappy-stubs-internal.o snappy-c.o
libtool: link: ranlib .libs/libsnappy.a
libtool: link: ( cd ".libs" && rm -f "libsnappy.la" && ln -s "../libsnappy.la" "libsnappy.la" )
c++ -DHAVE_CONFIG_H -I.      -fPIC -MT snappy_unittest-snappy_unittest.o -MD -MP -MF .deps/snappy_unittest-snappy_unittest.Tpo -c -o snappy_unittest-snappy_unittest.o `test -f 'snappy_unittest.cc' || echo './'`snappy_unittest.cc
mv -f .deps/snappy_unittest-snappy_unittest.Tpo .deps/snappy_unittest-snappy_unittest.Po
c++ -DHAVE_CONFIG_H -I.      -fPIC -MT snappy_unittest-snappy-test.o -MD -MP -MF .deps/snappy_unittest-snappy-test.Tpo -c -o snappy_unittest-snappy-test.o `test -f 'snappy-test.cc' || echo './'`snappy-test.cc
mv -f .deps/snappy_unittest-snappy-test.Tpo .deps/snappy_unittest-snappy-test.Po
/bin/sh ./libtool --tag=CXX    --mode=link c++   -fPIC   -o snappy_unittest snappy_unittest-snappy_unittest.o  snappy_unittest-snappy-test.o libsnappy.la -lz
libtool: link: c++ -fPIC -o .libs/snappy_unittest snappy_unittest-snappy_unittest.o snappy_unittest-snappy-test.o  ./.libs/libsnappy.so -lz -Wl,-rpath -Wl,/usr/local/lib
[build] Copying output files
[build] Copying the `build_detect_platform` template
[leveldb] Cleaning

make[1]: stopped in /root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18
[leveldb] Building command
[leveldb] Building

make[1]: stopped in /root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18
[leveldb] Build finished
[build] Copying output files

--- stderr
In file included from snappy_unittest.cc:39:
./snappy-test.h:135:20: warning: control reaches end of non-void function [-Wreturn-type]
  int Defaults() { }
                   ^
./snappy-test.h:161:3: warning: control may reach end of non-void function [-Wreturn-type]
  }
  ^
./snappy-test.h:179:3: warning: control may reach end of non-void function [-Wreturn-type]
  }
  ^
snappy_unittest.cc:165:1: warning: control may reach end of non-void function [-Wreturn-type]
}
^
snappy_unittest.cc:959:24: warning: implicit conversion from 'int' to 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::value_type' (aka 'char') changes value from 128 to -128 [-Wconstant-conversion]
  compressed.push_back(128);
             ~~~~~~~~~ ^~~
snappy_unittest.cc:960:24: warning: implicit conversion from 'int' to 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::value_type' (aka 'char') changes value from 128 to -128 [-Wconstant-conversion]
  compressed.push_back(128);
             ~~~~~~~~~ ^~~
snappy_unittest.cc:961:24: warning: implicit conversion from 'int' to 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::value_type' (aka 'char') changes value from 128 to -128 [-Wconstant-conversion]
  compressed.push_back(128);
             ~~~~~~~~~ ^~~
snappy_unittest.cc:962:24: warning: implicit conversion from 'int' to 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::value_type' (aka 'char') changes value from 128 to -128 [-Wconstant-conversion]
  compressed.push_back(128);
             ~~~~~~~~~ ^~~
snappy_unittest.cc:963:24: warning: implicit conversion from 'int' to 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::value_type' (aka 'char') changes value from 128 to -128 [-Wconstant-conversion]
  compressed.push_back(128);
             ~~~~~~~~~ ^~~
9 warnings generated.
In file included from snappy-test.cc:31:
./snappy-test.h:135:20: warning: control reaches end of non-void function [-Wreturn-type]
  int Defaults() { }
                   ^
./snappy-test.h:161:3: warning: control may reach end of non-void function [-Wreturn-type]
  }
  ^
./snappy-test.h:179:3: warning: control may reach end of non-void function [-Wreturn-type]
  }
  ^
3 warnings generated.
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 19: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 21: Could not find build_config.mk
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 36: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 38: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 74: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 76: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 81: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 93: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 98: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 198: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 221: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 227: Need an operator
make[1]: Fatal errors encountered -- cannot continuemake[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 19: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 21: Could not find build_config.mk
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 36: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 38: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 74: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 76: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 81: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 93: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 98: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 198: Missing dependency operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 221: Need an operator
make[1]: "/root/.cargo/registry/src/github.com-1ecc6299db9ec823/leveldb-sys-2.0.1/deps/leveldb-1.18/Makefile" line 227: Need an operator
make[1]: Fatal errors encountered -- cannot continuethread 'main' panicked at 'copy of output files failed', src/libcore/option.rs:1034:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::option::expect_failed
   9: core::option::Option<T>::expect
             at /wrkdirs/usr/ports/lang/rust/work/rustc-1.35.0-src/src/libcore/option.rs:312
  10: build_script_build::build_leveldb
             at ./src/build.rs:115
  11: build_script_build::main
             at ./src/build.rs:178
  12: std::rt::lang_start::{{closure}}
             at /wrkdirs/usr/ports/lang/rust/work/rustc-1.35.0-src/src/libstd/rt.rs:64
  13: std::panicking::try::do_call
  14: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  15: std::rt::lang_start_internal
  16: std::rt::lang_start
             at /wrkdirs/usr/ports/lang/rust/work/rustc-1.35.0-src/src/libstd/rt.rs:64
  17: main
  18: _start
  19: <unknown>

warning: build failed, waiting for other jobs to finish...
error: build failed
gmake: *** [Makefile:22: build] Error 101

[binarylogic]

Hi @sjolicoeur thanks for this! And yes, we'd love to support the BSD family. We're working on expanding our support as much as possible. Vector depends on leveldb and rdkafka since they are both very stable and reliable, unfortunately this makes compilation difficult (as you pointed out above).

@a-rodin was able to, impressively, compile Vector by statically linking all dependencies in #689. You might be able to use that for inspiration. Otherwise I can start to look for someone that can help.

[valpackett]

Looks like there were related changes in leveldb-sys recently, have you tried replacing the release version of leveldb deps with the git one?

[LucioFranco]

A workaround currently for FreeBSD support is to compile vector with this command:

$ cargo build --release --no-default-features

This will build a version of vector without any of the C/C++ depedencies. This means there would be no kafka sink and no disk buffering.

[valpackett]

leveldb-sys 2.0.4 should work

[sjolicoeur]

Apologies for the delay I tried cargo build --release --no-default-features but to no avail

[root@ev14 /tmp/vector/vector-master]# export RUST_BACKTRACE=1
[root@ev14 /tmp/vector/vector-master]# cargo build --release --no-default-features
   Compiling toml v0.4.10
   Compiling hotmic v0.8.2
   Compiling http v0.1.17
   Compiling file-source v0.1.0 (/tmp/vector/vector-master/lib/file-source)
   Compiling tracing-fmt v0.1.0 (https://github.com/tokio-rs/tracing#0a5ba16c)
   Compiling typetag v0.1.3
   Compiling tokio-reactor v0.1.9
   Compiling tokio-codec v0.1.1
   Compiling tower-util v0.1.0
   Compiling h2 v0.1.21
   Compiling http-body v0.1.0
   Compiling headers-core v0.1.0
   Compiling tracing-metrics v0.1.0 (/tmp/vector/vector-master/lib/tracing-metrics)
   Compiling tokio-fs v0.1.6
   Compiling prost-types v0.4.0
   Compiling prost v0.5.0
   Compiling string_cache v0.7.3
   Compiling codec v0.1.0 (/tmp/vector/vector-master/lib/codec)
   Compiling built v0.3.0
   Compiling tower v0.1.0 (https://github.com/tower-rs/tower.git#40fbb85c)
   Compiling tower v0.1.1
   Compiling tokio-tcp v0.1.3
   Compiling tokio-udp v0.1.3
   Compiling tokio-uds v0.2.5
   Compiling tokio-signal v0.2.7
   Compiling headers v0.2.1
   Compiling prost-build v0.4.0
   Compiling prost-types v0.5.0
error: failed to run custom build command for `prost-build v0.4.0`
process didn't exit successfully: `/tmp/vector/vector-master/target/release/build/prost-build-dff63faaeaade81f/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Failed to find the protoc binary. The PROTOC environment variable is not set, there is no bundled protoc for this platform, and protoc is not in the PATH', src/libcore/option.rs:1034:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::option::expect_failed
   9: build_script_build::main
  10: std::rt::lang_start::{{closure}}
  11: std::panicking::try::do_call
  12: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  13: std::rt::lang_start_internal
  14: main
  15: _start
  16: <unknown>

warning: build failed, waiting for other jobs to finish...
error: build failed

adding leveldb-sys = "2.0.4" to Cargo.toml resulted in:

[root@ev14 /tmp/vector/vector-master]# cargo build --release --no-default-features
    Updating crates.io index
 Downloading crates ...
  Downloaded leveldb-sys v2.0.4
   Compiling leveldb-sys v2.0.4
   Compiling prost-build v0.4.0
   Compiling tokio-process v0.2.3
   Compiling http-connection v0.1.0
error: failed to run custom build command for `prost-build v0.4.0`
process didn't exit successfully: `/tmp/vector/vector-master/target/release/build/prost-build-dff63faaeaade81f/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Failed to find the protoc binary. The PROTOC environment variable is not set, there is no bundled protoc for this platform, and protoc is not in the PATH', src/libcore/option.rs:1034:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::option::expect_failed
   9: build_script_build::main
  10: std::rt::lang_start::{{closure}}
  11: std::panicking::try::do_call
  12: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  13: std::rt::lang_start_internal
  14: main
  15: _start
  16: <unknown>

warning: build failed, waiting for other jobs to finish...
error: build failed

[valpackett]

@sjolicoeur the message is telling you to install protobuf

[sjolicoeur]

installed pkg install protobuf version 3.7.1. And... Great news! It compiled! Less great news it is giving me an error:

[root@ev14 /tmp/vector/vector-master]# cat simple.tomml
[sources.my_source_id]
  # REQUIRED - General
  type = "stdin" # must be: "stdin"

  # OPTIONAL - General
  max_length = 102400 # default, bytes

  # OPTIONAL - Context
  host_key = "host" # default

[root@ev14 /tmp/vector/vector-master]# ./target/debug/vector -c simple.tomml
Sep 01 21:45:11.088  INFO vector: Log level "info" is enabled.
Sep 01 21:45:11.088  INFO vector: Loading config. path="simple.tomml"
Sep 01 21:45:11.128 ERROR log: Configuration error: unknown variant `stdin`, there are no variants for key `sources.my_source_id.type`  log.target="vector" log.module_path="vector" log.file="src/main.rs" log.line=292
Sep 01 21:45:11.128 ERROR vector: Configuration error: unknown variant `stdin`, there are no variants for key `sources.my_source_id.type`

is there a way to list the supported variants from the command line?

[LucioFranco]

@sjolicoeur odd, could you paste what commit sha you are on?

[ghost]

Could it be the same issue as in #791? The problem there was mitigated by use of GNU ld instead of LLVM lld (#794). Probably the linker used here is LLVM lld too.

Assuming that the system C compiler is clang, the linker used by Rust can be changed to the GNU one by exporting RUSTFLAGS=-Clink-arg=-fuse-ld=/usr/local/bin/ld, where /usr/local/bin/ld is the path to GNU ld.

[valpackett]

The system linker is indeed LLD on modern FreeBSD.

You don't need the full path btw, just -fuse-ld=bfd to use the (ancient) GNU BFD ld in base.

// Is this a bug in LLD or in typetag? Does the LLD team know about this?

[ghost]

Is this a bug in LLD or in typetag? Does the LLD team know about this?

It might be related to this issue. But if it is an LLD and not Rust compiler problem, then I think it should be isolated (i.e. reproduced without Rust) before reporting.

[shaunren]

Same issue here on OpenBSD 6.6. Using GNU ld didn't help either.

[phyber]

Seems to build fine for me in debug mode on FreeBSD 12.0-RELEASE-p10 (cargo build). The tests also build and run:

$ cargo test
    Finished dev [unoptimized + debuginfo] target(s) in 0.92s
     Running target/debug/deps/vector-20c2206c9ba50113

running 270 tests
test event::metric::test::merge_counters ... ok
test event::metric::test::merge_gauges ... ok
test event::metric::test::merge_histograms ... ok
test buffers::test::drop_when_full ... ok
test event::metric::test::merge_incompatible_counters ... ok
test event::metric::test::merge_sets ... ok
test event::test::event_iteration ... ok
test event::test::type_serialization ... ok
test event::test::serialization ... ok
test event::unflatten::tests::merge_array ... ok
test event::unflatten::tests::nested ... ok
test event::unflatten::tests::nested_array ... ok
test event::unflatten::tests::array ... ok
test region::tests::region_custom_name_endpoint ... ok
test region::tests::region_es_east_1 ... ok
test region::tests::region_not_provided ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... error: process didn't exit successfully: `/usr/home/foobar/code/vector/target/debug/deps/vector-20c2206c9ba50113` (signal: 11, SIGSEGV: invalid memory reference)

That last test is a little worrying though.

[LucioFranco]

@phyber that is one interesting error...possibly a serde_json bug?

[phyber]

Bumping serde_json to the latest version doesn't fix it, but I've just realised that the output here is random, different runs will crash in different places.

running 270 tests
test event::metric::test::merge_counters ... ok
test event::metric::test::merge_gauges ... ok
test event::metric::test::merge_histograms ... ok
test buffers::test::drop_when_full ... ok
test event::metric::test::merge_incompatible_counters ... ok
test event::metric::test::merge_sets ... ok
test event::test::event_iteration ... ok
test event::test::type_serialization ... ok
test event::test::serialization ... ok
test event::unflatten::tests::merge_array ... ok
test event::unflatten::tests::nested ... ok
test event::unflatten::tests::nested_array ... ok
test event::unflatten::tests::array ... ok
test region::tests::region_custom_name_endpoint ... ok
test region::tests::region_es_east_1 ... ok
test region::tests::region_not_provided ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... ok
error: process didn't exit successfully: `/usr/home/foobar/code/vector/target/debug/deps/vector-8b367f4a56112f25` (signal: 11, SIGSEGV: invalid memory reference)

It's likely a random test is segfaulting, since the tests aren't run in any specific order. I'll try to narrow this down later by removing #[test]s until I find the one that's crashing.

[ghost]

@phyber
Please try running tests with --test-threads=1 option, i.e.

cargo test -- --test-threads 1

which would run tests serially and thus the latest printed test would be the one causing segmentation fault.

I've observed grok_parser tests causing segmentation fault on armv7-unknown-linux-musleabihf (see this comment), which might be related.

[phyber]

OK, running with a single thread I see

running 270 tests
test buffers::test::drop_when_full ... ok
test event::metric::test::merge_counters ... ok
test event::metric::test::merge_gauges ... ok
test event::metric::test::merge_histograms ... ok
test event::metric::test::merge_incompatible_counters ... ok
test event::metric::test::merge_sets ... ok
test event::test::event_iteration ... ok
test event::test::serialization ... ok
test event::test::type_serialization ... ok
test event::unflatten::tests::array ... ok
test event::unflatten::tests::merge_array ... ok
test event::unflatten::tests::nested ... ok
test event::unflatten::tests::nested_array ... ok
test event::unflatten::tests::unflatten_abirtrary ... ok
test region::tests::region_custom_name_endpoint ... ok
test region::tests::region_es_east_1 ... ok
test region::tests::region_not_provided ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... error: process didn't exit successfully: `/usr/home/foobar/code/vector/target/debug/deps/vector-8b367f4a56112f25 --test-threads 1 --nocapture` (signal: 11, SIGSEGV: invalid memory reference)

Most of the time, however, we occasionally get slightly further:

running 270 tests
test buffers::test::drop_when_full ... ok
test event::metric::test::merge_counters ... ok
test event::metric::test::merge_gauges ... ok
test event::metric::test::merge_histograms ... ok
test event::metric::test::merge_incompatible_counters ... ok
test event::metric::test::merge_sets ... ok
test event::test::event_iteration ... ok
test event::test::serialization ... ok
test event::test::type_serialization ... ok
test event::unflatten::tests::array ... ok
test event::unflatten::tests::merge_array ... ok
test event::unflatten::tests::nested ... ok
test event::unflatten::tests::nested_array ... ok
test event::unflatten::tests::unflatten_abirtrary ... ok
test region::tests::region_custom_name_endpoint ... ok
test region::tests::region_es_east_1 ... ok
test region::tests::region_not_provided ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encoded_event_retains_timestamp ... error: process didn't exit successfully: `/usr/home/foobar/code/vector/target/debug/deps/vector-8b367f4a56112f25 --test-threads 1` (signal: 11, SIGSEGV: invalid memory reference)

[LucioFranco]

btw al the encode tests rely on serde_json and chrono, nothing to do with actually talking to cloudwatch.

[ghost]

One way to find out what exactly code is causing the segmentation fault is to use a debugger to print backtrace.

Using LLDB it can be done as

lldb -o run -o 'thread backtrace' -- /usr/home/foobar/code/vector/target/debug/deps/vector-8b367f4a56112f25 sinks::aws_cloudwatch_logs::tests

or using GDB as

gdb -q -ex 'set print thread-events off' -ex run -ex backtrace --args /usr/home/foobar/code/vector/target/debug/deps/vector-8b367f4a56112f25 sinks::aws_cloudwatch_logs::tests

[phyber]

For anyone else unfamiliar with debugging this on FreeBSD, you'll either need to run the debugger as root, or enable debugging for unprivileged users with sysctl -w security.bsd.unprivileged_proc_debug=1.

Running this with lldb, I get the following:

running 8 tests
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event_with_postfix ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event_with_prefix ... ok
test sinks::aws_cloudwatch_logs::tests::partition_no_key_event ... ok
test sinks::aws_cloudwatch_logs::tests::partition_static ... ok
Process 53989 stopped
* thread #29, name = 'hyper-dns0', stop reason = signal SIGSEGV: invalid address (fault address: 0x370)
    frame #0: 0x0000000804f1b7f4 libc.so.7`___lldb_unnamed_symbol256$$libc.so.7 + 148
libc.so.7`___lldb_unnamed_symbol256$$libc.so.7:
->  0x804f1b7f4 <+148>: movq   %rbx, 0x370(%rcx)
    0x804f1b7fb <+155>: movq   %rbx, 0x378(%rax)
    0x804f1b802 <+162>: movq   0x370(%rbx), %rax
    0x804f1b809 <+169>: jmp    0x804f1b80e               ; <+174>

Another run:

running 8 tests
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event_with_postfix ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event_with_prefix ... ok
test sinks::aws_cloudwatch_logs::tests::partition_no_key_event ... ok
test sinks::aws_cloudwatch_logs::tests::partition_static ... ok
Process 54189 stopped
* thread #34, name = 'hyper-dns1', stop reason = signal SIGSEGV: invalid address (fault address: 0x370)
    frame #0: 0x0000000804f1b7f4 libc.so.7`___lldb_unnamed_symbol256$$libc.so.7 + 148
libc.so.7`___lldb_unnamed_symbol256$$libc.so.7:
->  0x804f1b7f4 <+148>: movq   %rbx, 0x370(%rcx)
    0x804f1b7fb <+155>: movq   %rbx, 0x378(%rax)
    0x804f1b802 <+162>: movq   0x370(%rbx), %rax
    0x804f1b809 <+169>: jmp    0x804f1b80e               ; <+174>

A successful run!

running 8 tests
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event_with_postfix ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event_with_prefix ... ok
test sinks::aws_cloudwatch_logs::tests::partition_no_key_event ... ok
test sinks::aws_cloudwatch_logs::tests::partition_static ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encoded_event_retains_timestamp ... ok

test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 265 filtered out

Failure earlier on:

running 8 tests
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_text ... ok
test sinks::aws_cloudwatch_logs::tests::cloudwatch_encode_log_as_json ... ok
test sinks::aws_cloudwatch_logs::tests::partition_event ... ok
Process 54318 stopped
* thread #38, name = 'hyper-dns0', stop reason = signal SIGSEGV: invalid address (fault address: 0x370)
    frame #0: 0x0000000804f1b7f4 libc.so.7`___lldb_unnamed_symbol256$$libc.so.7 + 148
libc.so.7`___lldb_unnamed_symbol256$$libc.so.7:
->  0x804f1b7f4 <+148>: movq   %rbx, 0x370(%rcx)
    0x804f1b7fb <+155>: movq   %rbx, 0x378(%rax)
    0x804f1b802 <+162>: movq   0x370(%rbx), %rax
    0x804f1b809 <+169>: jmp    0x804f1b80e               ; <+174>

The failures seem to always be in the hyper DNS threads, and it appears to be a race condition, given that it sometimes doesn't occur, and the switches between the threads that are crashing (first crash is hyper-dns0, second is hyper-dns1).

After I've updated to FreeBSD 12.1 in a while, I'll try getting the libc debug symbols onto the machine to hopefully get slightly nicer output.

[valpackett]

What does hyper use for DNS resolution? The system resolver, c-ares, something written in Rust?

[LucioFranco]

So I believe this is the call itself https://github.com/hyperium/hyper/blob/0.12.x/src/client/connect/dns.rs#L212 this may be a bug with rust itself since it looks like its just using https://doc.rust-lang.org/stable/std/net/trait.ToSocketAddrs.html

[ghost]

I think we are not going to support this anytime soon. There are the following complications:

• No BSD operating system has tier 1 support in Rust: https://forge.rust-lang.org/release/platform-support.html#tier-1.
• There are difficulties with providing bindings to native dependencies, as the usual way to deal with this in Rust is to bundle prepared C bindings, which implies stable ABI. See How to deal with breaking changes on platform ? [BSDs related] rust-lang/libc#570 for details.

[valpackett]

we are not going to support this anytime soon

Well, what does "support" even mean? :)

No one has asked for official binaries (we like to build packages ourselves) or any kind of guarantees, we're just here to discuss solutions to bugs.

No BSD operating system has tier 1 support in Rust

Note that Tier 2 includes official binary builds of Rust and all Rust commits being gated on these platforms building. Tier 1 is just the "extra special", the most mainstream platforms.

stable ABI

Note that FreeBSD keeps compatibility with old binaries around for ages (4.x binaries still work), the issue with the C bindings is with the compilation process (Rust libc getting outdated) and it can be "solved" by pinning symbol versions (linking the libc crate to e.g. 32-bit-inode versions of FS functions).

---

Looks like the dependency update fixing the LLD bug has landed, yay. I'm going to test on my machines now.

[valpackett]

On -current with rust nightly, the aws_cloudwatch_logs tests actually crash with a malloc assertion

<jemalloc>: Should own 0 locks of rank >= 1: tcache_ql(12)

from setting thread name

  * frame #0: 0x00000008056a1d23 libc.so.7`__je_tcache_arena_associate(tsdn=0x0000000807fa6090, tcache=0x0000000807fa6250, arena=0x0000000808200900) at jemalloc_tcache.c:296:3
    frame #1: 0x00000008056a4125 libc.so.7`arena_choose_impl(tsd=0x0000000807fa6090, arena=0x0000000000000000, internal=<unavailable>) at jemalloc_internal_inlines_b.h:0
    frame #2: 0x00000008056a2303 libc.so.7`__je_tsd_tcache_data_init [inlined] arena_choose(tsd=<unavailable>, arena=0x0000000000000000) at jemalloc_internal_inlines_b.h:63:9
    frame #3: 0x00000008056a22f9 libc.so.7`__je_tsd_tcache_data_init(tsd=0x0000000807fa6090) at jemalloc_tcache.c:419
    frame #4: 0x00000008056a21f0 libc.so.7`__je_tsd_tcache_enabled_data_init(tsd=0x0000000807fa6090) at jemalloc_tcache.c:348:3
    frame #5: 0x00000008056a026c libc.so.7`__je_tsd_fetch_slow(tsd=0x0000000807fa6090, minimal=<unavailable>) at jemalloc_tsd.c:0
    frame #6: 0x00000008056f23c5 libc.so.7`imalloc [inlined] tsd_fetch_impl at tsd.h:266:10
    frame #7: 0x00000008056f23ab libc.so.7`imalloc [inlined] tsd_fetch at tsd.h:292
    frame #8: 0x00000008056f23ab libc.so.7`imalloc(sopts=0x00007fffdfbfbd00, dopts=0x00007fffdfbfbcd0) at jemalloc_jemalloc.c:2003
    frame #9: 0x00000008056f2261 libc.so.7`__malloc(size=<unavailable>) at jemalloc_jemalloc.c:2042:2
    frame #10: 0x000000080569fe51 libc.so.7`strdup(str="hyper-dns2") at strdup.c:49:14
    frame #11: 0x000000080548e9dc libthr.so.3`_pthread_set_name_np + 188
    frame #12: 0x000000000534c244 vector-a2e8f0f0639438e0`std::thread::Builder::spawn_unchecked::_$u7b$$u7b$closure$u7d$$u7d$::hd41d55bfe3359ee0 at mod.rs:464:16

Googling the assertion revealed jemalloc/jemalloc#1234 (hah, wow, issue number 1-2-3-4)

aaand tests consistently pass with MALLOC_CONF=tcache:false set in the env (which apparently would be JE_MALLOC_CONF on platforms where jemalloc is not the one and only libc malloc).

Would be interesting to try on Linux with jemallocator!

[ghost]

Well, what does "support" even mean? :)

No one has asked for official binaries (we like to build packages ourselves) or any kind of guarantees, we're just here to discuss solutions to bugs.

Supported platforms have the following:

• CI tests which ensure that everything functions properly and new changes don't break the platform support. Note CircleCI currently supports only Linux, Windows, and macOS executors.
• Handy ways to install both stable and nightly versions of Vector. For supported platforms it does include pre-compiled binaries, as building Vector from source is time-consuming and requires Rust as a dependency.

So for supported platforms there are certain guarantees provided by the items listed above. Of course it doesn't prevent Vector from happening to work on some of other platforms too, but no claims about feature completeness or stability are made for them.

[valpackett]

Hmm! Actually turns out jemallocator is already used here, and it was used on FreeBSD too, and turning it off solves the problem.

[phyber]

Apologies for not getting around to performing the lldb steps on FreeBSD 12.1 yet, but I can at least confirm that disabling jemallocator also allows the tests to run successfully here, with all of them passing.

[valpackett]

Now you should be able to just pkg install vector :)

[Hoverbear]

@myfreeweb It works! It looks like it could use a update with 0.7, could you recommend where we can send a version bump?

[Hoverbear]

Looks like https://svnweb.freebsd.org/ports/head/sysutils/vector/pkg-descr?view=log has a contact for the maintainer.

[valpackett]

@Hoverbear that's me! :) I'll look into bumping the version soon.

[valpackett]

Port is at 0.7.1 now, binary packages should get built soon.

[Hoverbear]

@myfreeweb Thank you so much!

[binarylogic]

@Hoverbear can we close this as a result? And thank you @myfreeweb!