Install icu-full when building from source

[thom4parisot]

• Operating system and version: macOS Sierra

• nvm debug output:

nvm --version: v0.33.8
$TERM_PROGRAM: iTerm.app
$SHELL: /bin/zsh
$HOME: /Users/oncletom
$NVM_DIR: '$HOME/.nvm'
$PREFIX: ''
$NPM_CONFIG_PREFIX: ''
$NVM_NODEJS_ORG_MIRROR: ''
$NVM_IOJS_ORG_MIRROR: ''
shell version: 'zsh 5.2 (x86_64-apple-darwin16.0)'
uname -a: 'Darwin 16.7.0 Darwin Kernel Version 16.7.0: Thu Jan 11 22:59:40 PST 2018; root:xnu-3789.73.8~1/RELEASE_X86_64 x86_64'
OS version: Mac 10.12.6 16G1212
curl: /usr/bin/curl, curl 7.54.0 (x86_64-apple-darwin16.0) libcurl/7.54.0 SecureTransport zlib/1.2.8
wget: /usr/local/bin/wget, GNU Wget 1.19.4 built on darwin16.7.0.
git: /usr/local/bin/git, git version 2.16.1
grep: grep: aliased to grep --color=auto (grep --color=auto), grep (BSD grep) 2.5.1-FreeBSD
awk: /usr/bin/awk, awk version 20070501
sed: illegal option -- -
usage: sed script [-Ealn] [-i extension] [file ...]
       sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file ...]
sed: /usr/bin/sed,
cut: illegal option -- -
usage: cut -b list [-n] [file ...]
       cut -c list [file ...]
       cut -f list [-s] [-d delim] [file ...]
cut: /usr/bin/cut,
basename: illegal option -- -
usage: basename string [suffix]
       basename [-a] [-s suffix] string [...]
basename: /usr/bin/basename,
rm: illegal option -- -
usage: rm [-f | -i] [-dPRrvW] file ...
       unlink file
rm: /bin/rm,
mkdir: illegal option -- -
usage: mkdir [-pv] [-m mode] directory ...
mkdir: /bin/mkdir,
xargs: illegal option -- -
usage: xargs [-0opt] [-E eofstr] [-I replstr [-R replacements]] [-J replstr]
             [-L number] [-n number [-x]] [-P maxprocs] [-s size]
             [utility [argument ...]]
xargs: /usr/bin/xargs,
nvm current: system
which node: /usr/local/bin/node
which iojs: iojs not found
which npm: /usr/local/bin/npm
npm config get prefix: /usr/local
npm root -g: /usr/local/lib/node_modules

• nvm ls output:

         v9.3.0
->       system
default -> v8 (-> N/A)
node -> stable (-> v9.3.0) (default)
stable -> 9.3 (-> v9.3.0) (default)
iojs -> N/A (default)
lts/* -> lts/carbon (-> N/A)
lts/argon -> v4.8.7 (-> N/A)
lts/boron -> v6.12.3 (-> N/A)
lts/carbon -> v8.9.4 (-> N/A)

• How did you install nvm? (e.g. install script in readme, homebrew): homebrew

• What steps did you perform?

I tried to install node from source to specify additional ICU options.

nvm install -s v8 --with-intl=system-icu
nvm install -s v8 --with-intl=full-icu --download=all

• What happened?

Install failed. Although I specified either system-icu or full-icu, the logs refer to small-icu.

Detected that you have 4 CPU core(s)
Running with 3 threads to speed up the build
Additional options while compiling:  --with-intl=full-icu --download=all
Clang v3.5+ detected! CC or CXX not specified, will use Clang as C/C++ compiler!
Local cache found: $NVM_DIR/.cache/src/node-v8.9.4/node-v8.9.4.tar.xz
Checksums match! Using existing downloaded archive $NVM_DIR/.cache/src/node-v8.9.4/node-v8.9.4.tar.xz
$>./configure --prefix=/Users/oncletom/.nvm/versions/node/v8.9.4  --with-intl=full-icu --download=all<
creating icu_config.gypi
* Using ICU in deps/icu-small
creating icu_config.gypi
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'asan': 0,
                 'coverage': 'false',
                 'debug_devtools': 'node',
                 'debug_http2': 'false',
                 'debug_nghttp2': 'false',
                 'force_dynamic_crt': 0,
                 'host_arch': 'x64',
                 'icu_data_file': 'icudt59l.dat',
                 'icu_data_in': '../../deps/icu-small/source/data/in/icudt59l.dat',
                 'icu_endianness': 'l',
                 'icu_gyp_path': 'tools/icu/icu-generic.gyp',
                 'icu_locales': 'en,root',
                 'icu_path': 'deps/icu-small',
                 'icu_small': 'true',
                 'icu_ver_major': '59',
                 'llvm_version': 0,
                 'node_byteorder': 'little',
                 'node_enable_d8': 'false',
                 'node_enable_v8_vtunejit': 'false',
                 'node_install_npm': 'true',
                 'node_module_version': 57,
                 'node_no_browser_globals': 'false',
                 'node_prefix': '/Users/oncletom/.nvm/versions/node/v8.9.4',
                 'node_release_urlbase': '',
                 'node_shared': 'false',
                 'node_shared_cares': 'false',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_nghttp2': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_bundled_v8': 'true',
                 'node_use_dtrace': 'true',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'false',
                 'node_use_v8_platform': 'true',
                 'node_without_node_options': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'shlib_suffix': '57.dylib',
                 'target_arch': 'x64',
                 'uv_parent_path': '/deps/uv/',
                 'uv_use_dtrace': 'true',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 1,
                 'v8_enable_inspector': 1,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_promise_internal_field_count': 1,
                 'v8_random_seed': 0,
                 'v8_trace_maps': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 0,
                 'xcode_version': '9.0'}}
creating config.gypi
creating config.mk
gyp:  --with-intl=full-icu --download=all not found (cwd: /Users/oncletom/.nvm/.cache/src/node-v8.9.4/files) while trying to load  --with-intl=full-icu --download=all
Error running GYP
nvm: install v8.9.4 failed!

• What did you expect to happen?

To install node v8 with full ICU support.

• Is there anything in any of your profile files (.bashrc, .bash_profile, .zshrc, etc) that modifies the PATH?

[ljharb]

As you can see in ./configure --prefix=/Users/oncletom/.nvm/versions/node/v8.9.4 --with-intl=full-icu --download=all, the full-icu option is properly passed to configure; I'm not really sure what's going on here.

I tried reading through https://github.com/nodejs/node/wiki/Intl#full-icu-vs-small-icu but nothing popped up.

[rvagg]

$>./configure --prefix=/Users/oncletom/.nvm/versions/node/v8.9.4  --with-intl=full-icu --download=all<

Is the < at the end part of the command? or is that a bounding character like the $> at the start. It's a syntax error with it, but without it I don't get the same output and it does full-icu even with that same version of Node (at least from the full git source, maybe there's something about the tarball..). I wonder is there any persistent config files possible in this environment? Could there be a config.gypi, config.mk, etc. or is this going to be from a fresh source download?

[ljharb]

i assume it’s a bouncing character.

As for config files, would they all be relative to the node installation? Or might some be at home dir or machine level?

[thom4parisot]

Indeed, I did not realise bouncing chars were part of the pasted output.

I got puzzled when I saw * Using ICU in deps/icu-small. It seems flags are passed to the node configure command line. I wonder why it has no effect.

ICU configuration goes well when I run the same command with Node sources and with homebrew formula on macOS.

Is there anything that could be helpful to get down to the root of the problem?

[nickserv]

This is causing issues on Travis because it uses nvm to install Node, and some Node features work differently on Linux under the default ICU: travis-ci/travis-ci#9422

[ljharb]

@nickmccurdy fwiw, a test like that for number precision is inherently problematic, because precisions are always implementation-dependent - while this issue still needs to be solved in nvm, a better solution for you is to change your test to be robust against varying precisions.

[nickserv]

Thanks, I was using Number.prototype.toLocaleString but I couldn't figure out how to simulate a specific precision in my test so I removed it.

[AaronPresley]

I'm also having the same issue. When running:

nvm install -s v6.7.0 --with-intl=full-icu

I get the following output:

nvm install -s v6.7.0 --with-intl=full-icu
Additional options while compiling:  --with-intl=full-icu
Local cache found: $NVM_DIR/.cache/src/node-v6.7.0/node-v6.7.0.tar.xz
Checksums match! Using existing downloaded archive $NVM_DIR/.cache/src/node-v6.7.0/node-v6.7.0.tar.xz
$>./configure --prefix=/Users/apres2/.nvm/versions/node/v6.7.0  --with-intl=full-icu<
creating  ./icu_config.gypi
* Using ICU in deps/icu-small
creating  ./icu_config.gypi
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'asan': 0,
                 'debug_devtools': 'node',
                 'force_dynamic_crt': 0,
                 'host_arch': 'x64',
                 'icu_data_file': 'icudt57l.dat',
                 'icu_data_in': '../../deps/icu-small/source/data/in/icudt57l.dat',
                 'icu_endianness': 'l',
                 'icu_gyp_path': 'tools/icu/icu-generic.gyp',
                 'icu_locales': 'en,root',
                 'icu_path': 'deps/icu-small',
                 'icu_small': 'true',
                 'icu_ver_major': '57',
                 'llvm_version': 0,
                 'node_byteorder': 'little',
                 'node_enable_d8': 'false',
                 'node_enable_v8_vtunejit': 'false',
                 'node_install_npm': 'true',
                 'node_module_version': 48,
                 'node_no_browser_globals': 'false',
                 'node_prefix': '/Users/apres2/.nvm/versions/node/v6.7.0',
                 'node_release_urlbase': '',
                 'node_shared': 'false',
                 'node_shared_cares': 'false',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_bundled_v8': 'true',
                 'node_use_dtrace': 'true',
                 'node_use_etw': 'false',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'false',
                 'node_use_v8_platform': 'true',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'shlib_suffix': '48.dylib',
                 'target_arch': 'x64',
                 'uv_parent_path': '/deps/uv/',
                 'uv_use_dtrace': 'true',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 1,
                 'v8_inspector': 'true',
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 0,
                 'xcode_version': '9.0'}}
creating  ./config.gypi
creating  ./config.mk
gyp:  --with-intl=full-icu not found (cwd: /Users/apres2/.nvm/.cache/src/node-v6.7.0/files) while trying to load  --with-intl=full-icu
Error running GYP
nvm: install v6.7.0 failed!

Note that running nvm install -s v6.7.0 works as expected. This error only occurs when passing the --with-intl=full-icu option.

Using zsh 5.4.1 on macOS 10.12.6

Edit: For anyone still looking to solve this issue, I've written up some instructions on how to have multiple versions of node installed on your machine with full ICU support (without the use of nvm). See post here.

[thom4parisot]

Interestingly, the ICU download works when I change the following line:

https://github.com/creationix/nvm/blob/04c27e23fe82ff2edc7b3ff7f87fa191ad3e97e0/nvm.sh#L2060

with

./configure --prefix="${VERSION_PATH}" --with-intl=full-icu --download=all && \

I can't get my head around why the content of the variable is not received down the line by the ./configure command 🤔

PS: I use zsh with macOS 10.12.3 (16D30).

[ljharb]

@oncletom ok, this is getting somewhere - what happens if you echo "|$ADDITIONAL_PARAMETERS|" on the line before that? Then, what happens if you try it in bash? If it works in bash but not zsh, what's the output of setopt?

[thom4parisot]

This is the output I get:

| --with-intl=full-icu --download=all|

I tried replacing the ./configure line with that, and it worked (although it sounds like a hack):

./configure --prefix="${VERSION_PATH}" $(echo -n $ADDITIONAL_PARAMETERS) && \

This one works too:

command bash -c "./configure --prefix="${VERSION_PATH}" ${ADDITIONAL_PARAMETERS}" && \

$ setopt
alwaystoend
autocd
autopushd
autoresume
nobgnice
nocaseglob
cdablevars
nocheckjobs
noclobber
combiningchars
completeinword
correct
extendedglob
extendedhistory
noflowcontrol
histexpiredupsfirst
histfindnodups
histignorealldups
histignoredups
histignorespace
histsavenodups
histverify
nohup
incappendhistory
interactive
interactivecomments
login
longlistjobs
monitor
nonomatch
pathdirs
promptsubst
pushdignoredups
pushdsilent
pushdtohome
rcquotes
sharehistory
shinstdin
zle

When I do switch to bash, additional params works with the original code:

bash-3.2$ nvm install -s v10.0.0 --with-intl=full-icu --download=all
Detected that you have 4 CPU core(s)
Running with 3 threads to speed up the build
Additional options while compiling:  --with-intl=full-icu --download=all
Clang v3.5+ detected! CC or CXX not specified, will use Clang as C/C++ compiler!
Local cache found: $NVM_DIR/.cache/src/node-v10.0.0/node-v10.0.0.tar.xz
Checksums match! Using existing downloaded archive $NVM_DIR/.cache/src/node-v10.0.0/node-v10.0.0.tar.xz
$>./configure --prefix=/Users/oncletom/.nvm/versions/node/v10.0.0 --with-intl=full-icu --download=all<
setopt
creating icu_config.gypi
* ECMA-402 (Intl) support didn't find ICU in deps/icu..
 <https://ssl.icu-project.org/files/icu4c/61.1/icu4c-61_1-src.zip>

[ljharb]

Great - that means it’s specific to one of your enabled zsh options.

The easiest for me is if you binary search, disabling half of yours, trying again, etc, until you’ve narrowed down which option breaks it - at which point i can wrap the install command in code that disables that option.

[thom4parisot]

Well, I removed all but 3 setopt values (because zsh refuses to deactivate them):

$ setopt
interactive
shinstdin
zle

I still encounter the error.

BUT. It occurred to me the variable is well populated, it works in an echo command but not in a plain command. Also, the $VERSION_PATH variable is okay when it is used as an option value.

I found out the shwordsplit option enables a bash-like interpretation of variables. So setopt shwordsplit makes it work 👍

Although by reading
https://unix.stackexchange.com/questions/176326/running-commands-stored-in-shell-variables, I found out other ways of doing, like constructing $ADDITIONAL_PARAMETERS as an array instead of a string or the use of ${(s: :)ADDITIONAL_PARAMETERS} to force split the string into an array of parameters.

I feel the two latest ones are likely to have less/no side-effect on the rest of nvm.
What do you think?

[ljharb]

Unfortunately nvm isn’t bash, it’s posix, and there’s no arrays in posix.

My short term plan is to ensure shwordsplit is enabled around that command; if a better option presents itself I’m happy to use it :-)

[thom4parisot]

@ljharb cool no probs 🙂 Let met know if you need anything else. Happy to help review the fix when you feel like it.

[ljharb]

@oncletom when you have a chance, could you check out ljharb@3fc7b73 ? If that works for you in zsh, I'll merge it.

[thom4parisot]

@ljharb I just tested it and it's all good 🙂