please drop dependency (Imports) on V8 #831
Comments
makoshark
changed the title
|
For those that are in a similar situation, I've got a patched branch here that removes the dependency and the code that calls it and makes no other changes. You can install it with: I understand that this is not a real long-term solution but it will at least allow you to install the current version on systems that make installing V8 difficult or impossible.. |
|
Apologies you had to rummage through all this! But also do keep in mind the next version of the stan compiler needs v8. The new compiler is written in Ocaml and for R integration reasons is transpiled to javascript so rstan will need to use v8 in the future. v8 is currently used so we can have users test out Stan models and report back if their model doesn't work. That can be frustrating for users but the user reports have been very useful to us to make sure the new compiler is up to speed. |
|
Hopefully somebody can work out something for people running Stan on machines without root before the Stan3 compiler is released. A big challenge is that V8 upstream doesn't actually release buildable source so different distro have found a variety of ways or providing libv8. What Debian and Ubuntu do is just symlink NodeJS's I've been a Debian/Ubuntu developer for two decades and worked professionally doing this kind of integration/packaging and I failed to get an adequate |
|
The other interfaces (CmdStan,CmdStanR,CmdStanPy,PyStan) use stanc3 without requiring javascript. Will the use of javascript be dropped in the future, once stanc3 is verified as working on all models? |
|
@riddell-stan I hope so, but I don't know if @bgoodri has any alternative to using the javascript version of stanc3 if RStan needs to be on CRAN. @bgoodri can clarify further, but if I recall correctly CRAN can't handle the OCAML stuff. |
|
@makoshark I appreciate it (and all you've done for Debian over the years). I can put V8 into Suggests for the next rstan_2.21.x , but I can't get rstan in sync with upstream Stan 2.25+ in the foreseeable future without using javascript for the parser. As far as I can tell, the build dependencies for OCaml are much worse, especially on Windows where the makefiles have circular dependence with flexlink. So, I think we just have to help @jeroen get V8 to install everywhere. |
|
I've put V8 into Suggests in 8da6f56 so closing this issue, but if anyone has solutions to the V8 installation problems, let's discuss them on jeroen/V8#94 . |
|
The only non-javascript solution I've seen online is that we could use something like ocamlcc to translate from Ocaml -> C, then since the user wouldn't have the ocaml headers we would need to run the preprocessor to inline the |
|
Something like that was the pre-javascript plan. However, V8 is a very,
very widely used library with a very, very big company behind it, whereas
OCaml is the opposite of that. So, it seems that someone from that very,
very big company should help CRAN and Debian fix this problem, but until
that happens, I'll do what little I can.
…On Mon, Aug 31, 2020 at 4:32 PM Steve Bronder ***@***.***> wrote:
The only non-javascript solution I've seen online is that we could use
something like ocamlcc <https://github.com/bvaugon/ocamlcc> to translate
from Ocaml -> C, then since the user wouldn't have the ocaml headers we
would need to run the preprocessor to inline the #include <ocaml/*>
includes. Though this would be very difficult and I'm not really sure it
would work
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#831 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZ2XKSCGFGWJANE6FQXQLDSDQCGPANCNFSM4QERSEVQ>
.
|
|
I don't think there is any problem on Debian or Ubuntu or Fedora. The linked issue appears only on Gentoo because it doesn't have a suitable v8-devel package? |
|
I've copied the text of my reply over to a new comment at jeroen/V8#94 (comment) and edited this reply so it's just a pointer since I think that the V8 issue is the right place for this conversation to be happening. |
|
@makoshark this is getting really off-topic. A lot of R/python packages have system dependencies which you need to install with If you or your sysadmin doesn't want to do this, I guess you need something like linuxbrew or anaconda to install these libs in your home dir, and try to make that work with R. These systems are built exactly for this use case. Alternatively you need to build everything from source, but as you found out that is not always trivial, these days libraries are very advanced and require complex tooling to build. Another route you could go is do everything in a docker container (if your sysadmin will let you run docker at least). |
|
Neither R packages nor Python packages require installing system
dependencies with apt or yum.
Python packages, at least since the general adoption of the wheel
package format (ca. 2018), can ship platform-specific shared libraries
and executables inside packages. PyStan, for example, bundles the
platform-specific stanc3 binary inside the PyStan package which is
distributed via PyPI (Python's CRAN).
CRAN also allows for the distribution of platform-specific "binary
packages" which bundle dependencies along with the R package. This
eliminates the need to install dependencies separately, at least in theory.
…On 8/31/20 8:07 PM, Jeroen Ooms wrote:
@makoshark <https://github.com/makoshark> this is getting really
off-topic. A lot of R/python packages have system dependencies which you
need to install with |apt| or |yum| if you're on Linux. Installing v8 on
RHEL is as simple as |yum install v8-devel|.
If you or your sysadmin doesn't want to do this, I guess you need
something like linuxbrew or anaconda to install these libs in your home
dir, and try to make that work with R. These systems are built exactly
for this use case.
Alternatively you need to build everything from source, but as you found
out that is not always trivial, these days libraries are very advanced
and require complex tooling to build.
Another route you could go is do everything in a docker container (if
your sysadmin will let you run docker at least).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#831 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJQUBVVDQRSGF5OY4HGMITDSDQ3MDANCNFSM4QERSEVQ>.
|
As of today, github states: "This branch is 2 commits ahead, 48 commits behind stan-dev:develop. " - I am not able to realize how relevant the 48 commits behind are. Do you still think it is reasonable to use your patched package? Especially as the link jgabry gave states about v8:
|
|
The rstan github repo moves V8 to Suggests so you can just
remotes::install_github("stan-dev/rstan", ref = "develop", subdir =
"rstan/rstan")
But hopefully in a month or so, V8 is going to be a pretty hard
requirement, so start trying to get your sysadmins to administer their
systems ASAP.
…On Thu, Oct 22, 2020 at 11:56 AM gsgxnet ***@***.***> wrote:
For those that are in a similar situation, I've got a patched branch here
<https://github.com/makoshark/rstan> that removes the dependency and the
code that calls it and makes no other changes. You can install it with: install_github("makoshark/rstan",
ref="develop", subdir="rstan/rstan") with the devtools library.
I understand that this is not a real long-term solution but it will at
least allow you to install the current version on systems that make
installing V8 difficult or impossible..
As of today, github states: "This branch is 2 commits ahead, 48 commits
behind stan-dev:develop. " - I am not able to realize how relevant the 48
commits behind are. Do you still think it is reasonable to use your patched
package?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#831 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZ2XKQRPRE5SWIMS6OV3G3SMBIZ3ANCNFSM4QERSEVQ>
.
|
The problem with SUSE is worse. If there were any v8 development library package I would have installed it already. But looks like that package is not available in the SUSE universe any more since some years. I made a post in the openSUSE forums already, https://forums.opensuse.org/showthread.php/546178-R-essential-package-rstan-can-not-be-installed-because-libv8-devel-is-unavailable but no reply there so far. |
|
Can you install snaps?
https://snapcraft.io/install/node/opensuse
|
|
Jan helped by supplying a static v8 for the V8 R package. |
|
Awesome! You should try to get in touch with the OpenSuse folks to package this up, like for other linux distributions. |
|
The V8 R package now has an option to automatically download a static libv8 library when installing on Linux. To test: Sys.setenv(DOWNLOAD_STATIC_LIBV8 = 1)
install.packages("V8")This should help users that are on a Linux distribution that does not provide libv8. |
|
Seems to work. Thanks @jeroen |
|
Yes, it works! Now I have rstan on my Gentoo. Thanks, @jeroen ! |
|
This is on CRAN now. Hence as of V8 version 3.4.0 you can now also install V8 on any 64-bit Linux like this: Sys.setenv(DOWNLOAD_STATIC_LIBV8 = 1)
install.packages("V8") |
|
Thank You very much, @jeroen ! |
|
This is fantastic. This works great. I was able to get V8 to build from source following feedback from this (and several other) bugs but this is much easier and turns installation of this library on most HPC systems where users don't have room from a complicated and largely undocumented task into a very simple one. Thank you, @jeroen. |
|
I've written a short blog post about this new feature: https://ropensci.org/technotes/2020/11/12/installing-v8/ |
I get the error /usr/bin/ld: cannot find -lv8_monolith am I missing something? |
|
Hi Laura, you'll need to open an issue for this on the V8 github: https://github.com/jeroen/V8/issues |
@makoshark Thank you for a patched version, I could not make V8 on PowerPC work. However building from your repo ended up in error: Any advice will be greatly appreciated. |
|
|
@bgoodri Thank you for reply. Should I try to specify a path somewhere? (For sure make on its own works, I have build many things from sources starting from MacPorts and Git to R itself.) |
|
You can probably define MAKE inside ~/.R/Makevars
|
|
@bgoodri Got solved after specifying compilers for CXX14 and CXX17 and a full path to Make in Makevars. Honestly not sure what worked, as I changed these at once. Another error showed up, I found a fix, Rstan installed finally, however model compilation still fails with libtbb error: https://discourse.mc-stan.org/t/override-dependency-on-v8-or-make-v8-build-on-powerpc-mac-r-4-1-2/25432/ |
The current version of rstan includes a non-critical dependency on the V8 Javascript engine which is extremely difficult to install on some some systems.
libv8is available in most common Linux distributions but cannot be installed from within R and is very difficult to build from source.My lab runs rstan from a shared HPC super-computer where we do not have root/admin access. Installing the dependencies for the V8 package is extremely tricky because V8 itself does not release source and NodeJS (the most common distribution of V8) does not build a shared library version by default. Even when you build the shared library, it does not provide the required
libv8.soetc, so you need to create symlinks by hand (I had to look at the source code for the Debian package to figure this out).This is all extremely frustrating because rstan does not actually use Javascript for anything essential. The only place that the
v8()function is called when trying to download the nightly version of stan3 compiled to Javascript in a beta version of the function. This is also problematic because it assumes Internet access (we also don't have this in the HPC system).Basically, this dependency brings very little value while making the software difficult or impossible to install for those of us that do not have root or admin access on the research computing systems we rely on to use Stan.
The text was updated successfully, but these errors were encountered: