Failure to install packages with "sudo: no tty present and no askpass program specified" error

[mpinese]

I've run into an odd problem with building packages with apt dependencies, in which the password prompt for the sudo apt-get fails. At the moment a simple workaround is just installing the packages manually, but I'm posting what I've got in case you guys can see an easy fix.

The problem occurs on Lubuntu 15.10, when running julia 0.4.2 (staticfloat ppa release) from a local LXterminal session. I've observed it on Cairo.jl (_jl_libcairo error with Gadfly + Cairo #124 ) and HDF5.jl (see code block following).

julia> Pkg.build("HDF5")
INFO: Building Blosc
INFO: Building HDF5
Installing dependency hdf5-tools via `sudo apt-get install hdf5-tools`:
sudo: no tty present and no askpass program specified
================================[ ERROR: HDF5 ]=================================

LoadError: failed process: Process(`sudo apt-get install hdf5-tools`, ProcessExited(1)) [1]
while loading /home/mark/.julia/v0.4/HDF5/deps/build.jl, in expression starting on line 24

================================================================================

Hope that's of some use. Let me know if you need any more logs etc. -- I know Lubuntu isn't a very common distro! Thanks.

[zhenpan]

Hi Mark,

I got the same problem when Pkg.add("HDF5"). Could you please give more details about how "manually install the packages" ? Thank you.
Zhen

[yuyichao]

sudo apt-get install hdf5-tools

[zhenpan]

Yichao, thanks for the quick reply.

Here is the following problem. I found hdf5 was actually installed along with my anaconda, so I include the following command in .juliarc.jl

push!(Libdl.DL_LOAD_PATH, "/home/zhenpan/code/anaconda2/pkgs/hdf5-1.8.15.1-2/lib")

then

Pkg.build("HDF5")

signal (15): Terminated
while loading no file, in expression starting on line 0
syscall at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
uv__epoll_wait at /usr/bin/../lib/x86_64-linux-gnu/julia/libjulia.so (unknown line)
uv__io_poll at /usr/bin/../lib/x86_64-linux-gnu/julia/libjulia.so (unknown line)
uv_run at /usr/bin/../lib/x86_64-linux-gnu/julia/libjulia.so (unknown line)
process_events at ./stream.jl:709
wait at ./task.jl:346
wait at ./task.jl:277
jl_apply_generic at /usr/bin/../lib/x86_64-linux-gnu/julia/libjulia.so (unknown line)
anonymous at multi.jl:603
unknown function (ip: 0x7f61b581b569)
unknown function (ip: (nil))
unknown function (ip: (nil))
ERROR: Base.Pkg.PkgError("HDF5 is not an installed package")
[inlined code] from pkg.jl:13
in build! at pkg/entry.jl:506
in build! at pkg/entry.jl:551
[inlined code] from dict.jl:473
in build at pkg/entry.jl:573
in anonymous at pkg/dir.jl:31
in cd at file.jl:47
in cd at pkg/dir.jl:31
[inlined code] from pkg/dir.jl:25
in build at pkg.jl:52
in eval at ./boot.jl:263

Did I point julia to a wrong lib path?

[yuyichao]

The error suggests that you havent add HDF5 yet

[zhenpan]

Thanks, Yichao. Problem fixed.

After

sudo apt-get install hdf5-tools

and point julia to the system-wide hdf5 lib, and

Pkg.build("HDF5")

it works (though with another error in building Blosc, which was solved by Jingpeng [https://github.com/JuliaIO/HDF5.jl/issues/234] )

[nalimilan]

I'm seeing the same sudo error here on both 0.4.7 and master. Looks like it's not attached to the Julia terminal as it should, and therefore cannot ask for password.

EDIT: I'm on Fedora 24, so this isn't specific to apt-get distros.

EDIT2: run( sudo yum install hdf5 ) (with backticks) works, so the problem is specific to how BinDeps runs the command.

[abelsiqueira]

It looks like the problem happens with detach, which is called by Pkg.build.
I can reproduce this problem with

run(detach(`sudo ls`))

detach calls Cmd(..., detach=true), which is here. After that I can't understand.

[ihnorton]

Your analysis looks correct. This is a likely unintended side-effect of the fix for JuliaLang/julia#13458

We now have functionality to securely ask for passwords on all platforms (I think, see here and related links), so prompting could potentially be done by Julia. However, this would require some architectural changes, and Pkg is being redesigned right now anyway. For now the best option is probably to run installers manually before calling Pkg.build.

cc @StefanKarpinski -- not sure if this should go to the main Julia repo or the Pkg dev planner.

[nalimilan]

We could also use PackageKit (#66). pkcon install -y does not even require a password on my Fedora machine.

[Keno]

The problem is that detach no longer sets the controlling terminal. Blame @vtjnash ;)

[Keno]

See JuliaLang/julia#17904

[vtjnash]

This is a likely unintended side-effect of the fix for

FWIW, this was the intended effect. Our build slave should never become the controlling terminal master. Probably not a bad thing to prevent usage of stdin anyways, e.g. following the example of Debian (https://debian-handbook.info/browse/stable/sect.package-meta-information.html#sidebar.questions-conffiles#id-1.8.6.6.13.5).

[Keno]

Surely breaking BinDeps was not the intended effect. I also don't see what's wrong with passing the controlling terminal through to apt.

[SimonDanisch]

So, do I get this right, that the current fix is to install things manually?
This would make it difficult to give users a nice install experience.
Is fixing this on low priority?

[tkelman]

If someone can propose a fix we can try it. No one is actively working on this right now to my knowledge. I have plans to entirely replace bindeps in a way that doesn't rely on sudo or system package managers at all (at user Julia-package install time), but will certainly review any proposed changes here and try to keep it at least not getting any worse until a replacement is ready and widely usable (which is still a ways out).

[SimonDanisch]

So I was getting it to work by starting julia with sudo and passing -y to the apt-get command.... That'd be a bit better, but I heard people don't like either of these to be a default in a binary dependency installation...

[tkelman]

No, starting julia with sudo is a bad idea.

[SimonDanisch]

is there a way to figure out if we have the tty/askpass available, and if not replace the sudo command with an instruction?

[Keno]

The right solution is to pass through the controlling terminal as mentioned above.

[vtjnash]

if we have the tty

never / not not supposed to be possible

pass through the controlling terminal

no. we called detach specifically to prevent it from having a controlling terminal.

However, we could open a pseudo-tty pair (or use sudo -S and pass a normal pipe) and handle config IO ourselves.

[Keno]

no. we called detach specifically to prevent it from having a controlling terminal.

We called detach before that stopped giving it a controlling terminal. What would be the purpose of doing this other than breaking people's installation of packages.

[vtjnash]

I also don't see what's wrong with passing the controlling terminal through to apt.

The problem is you won't get it back. When Pkg.build finishes, it may kill all the other processes that were connected to that stdio (including the parent process and any other build processes).

[Keno]

The problem is you won't get it back. When Pkg.build finishes, it may kill all the other processes that were connected to that stdio (including the parent process and any other build processes).

Are you sure that's not a process-group related problem rather than a controlling terminal problem?

[vtjnash]

Yes. They get SIGPIPE not SIGHUP.

[Keno]

Can you elaborate by what mechanism you think this happens?

[Keno]

Certainly apt can't kill all processes with the same controlling terminal, because otherwise you couldn't use it from bash.

[vtjnash]

 Can you elaborate by what mechanism you think this happens?

by watching it happen in the debugger, and wondering why the whole process was crashing down without printing any error messages (because it couldn't - stderr was closed).

[vtjnash]

you can argue with the lawyer-ees all you will, but #199 (comment)

[Keno]

Ok, so your claim is that when a session leader exits, all file descriptors associated with that session leader's controlling terminal thereafter produce SIGPIPE?

[vtjnash]

Yes, that it what happened.

[vtjnash]

Also, the user loses the ability to interrupt or signal the host process (the REPL) via the keyboard as soon as the controlling terminal is lost.

[Keno]

Sorry, made a mistake, hold on.

[Keno]

Actually, I'm confused. The terminal should already be the controlling terminal of some other process group. Why did it used to get stolen at all?

[vtjnash]

Why did it used to get stolen at all?

It didn't, except for a couple of months on master. There was a libuv bug I accidentally didn't prevent from becoming a Julia bug when moving from v0.9 -> v1.7 (I was aware of it before then, which is why it wasn't a bug in our fork of v0.9, but mistakenly thought I could drop the revert commit during the rebase).

[Keno]

Ok, but this worked at some point after we started detaching, but before we stopped setting ctty, so somehow it got a controlling terminal.

[vtjnash]

Also, I'm not sure what you're end goal is here. That we should badly emulate the effect of not calling detach, while still calling detach?

[vtjnash]

Ok, but this worked at some point after we started detaching, but before we stopped setting ctty, so somehow it got a controlling terminal.

Yes, that's when / how I noticed the libuv bug had been introduced to master.

[Keno]

The goal is for sudo to be able to get at the controlling terminal so it can ask the user for a password.

[Keno]

Yes, that's when / how I noticed the libuv bug had been introduced to master.

I don't understand this mechanism though. Why was that terminal not already associated with a process group?

[vtjnash]

If that's the goal, then Pkg.build needs to stop being the daemon it was implemented to be (JuliaLang/julia#13506). Daemon's aren't supposed to be the controlling leader, or they wouldn't be daemons.

[Keno]

I thought somebody had good reasons for putting detach there. If not, then yes, it should be removed.

[vtjnash]

Why was that terminal not already associated with a process group?

I don't understand the question. It was associated with the process leader (Julia). The detached child process (libuv) was explicitly taking steps to ensure that it would steal the process leader away from its parent.

[Keno]

ctty stealing is not allowed unless you're root on linux.

[vtjnash]

While you're writing your fancy C programs, I would perhaps be remiss if I failed to point out that the original submitter of the bug report noting the breakage caused by libuv was a guy named loladiro (JuliaLang/julia#16398). 😉

[Keno]

That's under lldb IIRC

[haberdashPI]

In reading through this thread again I find myself very confused.

Is this affecting all binary dependencies on Ubuntu? How is, for example, Gtk.jl, successfully installed via its build.jl script (it's clear that it is)? I've tried to port my package to linux but have run into this problem. (Granted I am doing so via a VirtualBox install of Ubuntu, so maybe there is something specific to my environment that I'm running into...??).

[tkelman]

PkgEval gets around this by setting apt-get to always install things without bothering to ask for confirmation: https://github.com/JuliaCI/PackageEvaluator.jl/blob/60823fb6eebc69830368aa2a8ea6c84b26722695/scripts/setup.sh#L19-L27

[haberdashPI]

Got it! Thanks, that makes more sense.

[cstjean]

I'm consistently getting this error on travis, but only on Linux. It was working fine five days ago. I don't think I've made any change to my package that would affect this. Did anything else change?

[iblislin]

@cstjean I think your issue is unrelated.

the passed build is on precise

the failed build is on trusty which is the container-based runner. IIRC, sudo is not allowed on it.