Mac/Ubuntu Installation Fails syntax error near unexpected token PROTOBUF,'`

[PetrochukM]

Out:

$ brew install protobuf autoconf automake libtool
Warning: protobuf 3.3.2 is already installed
Warning: autoconf 2.69 is already installed
Warning: automake 1.15.1 is already installed
Warning: libtool 2.4.6_1 is already installed
$ ./autogen.sh
Running aclocal ...
Running autoheader...
Running libtoolize ..
Running automake ...
Running autoconf ...
$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking build system type... x86_64-apple-darwin16.6.0
checking host system type... x86_64-apple-darwin16.6.0
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 196608
checking how to convert x86_64-apple-darwin16.6.0 file names to x86_64-apple-darwin16.6.0 format... func_convert_file_noop
checking how to convert x86_64-apple-darwin16.6.0 file names to toolchain format... func_convert_file_noop
checking for /Library/Developer/CommandLineTools/usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking for ar... ar
checking for archiver @FILE support... no
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dsymutil... dsymutil
checking for nmedit... nmedit
checking for lipo... lipo
checking for otool... otool
checking for otool64... no
checking for -single_module linker flag... yes
checking for -exported_symbols_list linker flag... yes
checking for -force_load linker flag... yes
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... yes
checking for gcc option to produce PIC... -fno-common -DPIC
checking if gcc PIC flag -fno-common -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin16.6.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /Library/Developer/CommandLineTools/usr/bin/ld
checking if the linker (/Library/Developer/CommandLineTools/usr/bin/ld) is GNU ld... no
checking whether the g++ linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking for g++ option to produce PIC... -fno-common -DPIC
checking if g++ PIC flag -fno-common -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/Library/Developer/CommandLineTools/usr/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... darwin16.6.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking dependency style of gcc... (cached) gcc3
./configure: line 17067: syntax error near unexpected token `PROTOBUF,'
./configure: line 17067: `PKG_CHECK_MODULES(PROTOBUF, protobuf >= 2.4.0)'

Brew protobuf info:

Michaels-MacBook-Pro:sentencepiece petrochuk$ brew info protobuf
protobuf: stable 3.3.2 (bottled), HEAD
Protocol buffers (Google's data interchange format)
https://github.com/google/protobuf/
/usr/local/Cellar/protobuf/3.3.2 (260 files, 16.1MB) *
  Poured from bottle on 2017-08-02 at 17:17:01
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/protobuf.rb
==> Dependencies
Build: autoconf ✔, automake ✔, libtool ✔
==> Requirements
Optional: python3 ✔
==> Options
--with-python3
	Build with python3 support
--with-test
	Run build-time check
--without-python
	Build without python support
--HEAD
	Install HEAD version
==> Caveats
Editor support and examples have been installed to:
  /usr/local/opt/protobuf/share/doc/protobuf

Python modules have been installed and Homebrew's site-packages is not
in your Python sys.path, so you will not be able to import the modules
this formula installed. If you plan to develop with these modules,
please run:
  mkdir -p /Users/petrochuk/Library/Python/2.7/lib/python/site-packages
  echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/petrochuk/Library/Python/2.7/lib/python/site-packages/homebrew.pth

[PetrochukM]

Was able to resolve by following this comment thread:
stlink-org/stlink#83

But then ran into another error:

Michaels-MacBook-Pro:sentencepiece petrochuk$ make
cd . && /bin/sh ./config.status config.h
config.status: creating config.h
config.status: config.h is unchanged
/Library/Developer/CommandLineTools/usr/bin/make  all-recursive
Making all in src
/usr/local/opt/protobuf/share/doc/protobuf/bin/protoc --cpp_out=. ./sentencepiece.proto
make[2]: /usr/local/opt/protobuf/share/doc/protobuf/bin/protoc: No such file or directory
make[2]: *** [sentencepiece.pb.cc] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

[PetrochukM]

Resolved with
ln -s /usr/local/bin/protoc /usr/local/opt/protobuf/share/doc/protobuf/bin/protoc

PASS: spm_test
============================================================================
Testsuite summary for sentencepiece 0.1.0
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

[PetrochukM]

Another problem:

Michaels-MacBook-Pro:sentencepiece petrochuk$ sudo ldconfig -v
sudo: ldconfig: command not found

Resolved: protocolbuffers/protobuf#2570

[PetrochukM]

Ditto same first error for Ubuntu 16.04.

Installing pkg-config helped.

[taku910]

Seems that pkg-config helped to resolve this issue. Please let me close this issue.

[jcomfort4]

@taku910 @Deepblue129 This original issue is still occurring for me:

./configure: line 17073: syntax error near unexpected token `PROTOBUF,'
./configure: line 17073: `PKG_CHECK_MODULES(PROTOBUF, protobuf >= 2.4.0)'

It is not clear to me what part of the thread resolved this issue. Could you provide some more specificity around how you resolved it?

[jcomfort4]

To confirm, I have installed pkg-config, but this only seemed to resolve the second issue. It appears that the original issue is still unclear how to resolve it.

[PetrochukM]

Hi Friends!

This repo is old and has far to many problems and dependencies.

Built a gist for subwords in pure python from Google's new repo Tensor2Tensor.
https://gist.github.com/Deepblue129/a51defb2dc9506945f58a165026d1a96

I suggest you use that!

Thanks!

[jhsiao22]

I am working on a Mac running macOS Sierra. I also ran into difficulties when trying to install this software. Below was the error message I got after running the following commands.

cd /path/to/sentencepiece
./autogen.sh
./configure

Error Message

./configure: line 17073: syntax error near unexpected token `PROTOBUF,'
./configure: line 17073: `PKG_CHECK_MODULES(PROTOBUF, protobuf >= 2.4.0)'

In order to fix the problem, these are the commands that I had to do:

export PROTOBUF=/usr/local
export PROTOC="$PROTOBUF/bin/protoc"
export PROTOBUF_LIBS="-L$PROTOBUF/lib -lprotobuf -D_THREAD_SAFE"
export PROTOBUF_CFLAGS="-I$PROTOBUF/include -D_THREAD_SAFE" 

From what I can understand, I just needed to point the PROTOBUF variable to the correct library. I also believe I installed pkg-config, but I cannot remember if that explicitly helped the problem or not.

Lastly, I also ran into the "sudo ldconfig -v" problem, but is solved/discussed in another thread.

Hope this helps!