No output written from inside zsh <() construct

[a-j-wood]

Date: Thu, 10 Dec 2015 23:12:24 -0800
From: Frederik
Subject: problem with <(...)

Hello!

I just tried using 'pv' for the first time today - I have a
long-running script and I put 'pv' all sorts of pipelines.

Then I noticed that it was taking much longer than expected, and
hanging where it shouldn't hang...

A friend described 'pv' as a drop-in replacement for 'cat', but under
zsh, the following seems to hang forever:

join /usr/share/dict/words <(pv /usr/share/dict/words)

while the same command with 'cat' instead of 'pv' works fine. I
haven't investigated the other apparent performance issues, because
for me this surprise is enough to make me stop using 'pv'...

I guess I'd appreciate if the manual page were updated with this
information, because it took me quite some time to debug, and I think
other users should be spared the hassle.

Thanks for trying to make a great program!

Frederick

P.S. I think I'll go back to my homespun script, which seems less
likely to cause such problems: http://ix.io/mK3

---

Date: Mon, 14 Dec 2015 12:52:26 +0000
From: Andrew Wood
To: frederik
Subject: Re: problem with <(...) (PV)

The command you give works for me in bash.

Which version of pv and zsh are you running, and on which architecture?

---

Date: Mon, 14 Dec 2015 20:13:50 -0800
From: Frederik
To: Andrew Wood
Subject: Re: problem with <(...) (PV)

pv 1.6.0
zsh 5.2
Linux thutmose 4.2.5-1-ARCH #1 SMP PREEMPT Tue Oct 27 08:13:28 CET 2015 x86_64 GNU/Linux
(Arch is the distribution)

Same problem on

pv 1.2.0
zsh 4.3.17
Linux ofb.net 3.2.45-ofb #1 SMP Wed May 15 16:25:48 PDT 2013 i686 GNU/Linux
(Debian)

strace shows it as being stopped by SIGTTOU. Thanks! :)

---

Date: Wed, 16 Dec 2015 12:09:12 +0000
From: Andrew Wood
To: Frederik
Subject: Re: problem with <(...) (PV)

Thanks.

It looks like zsh starts <() processes disconnected from the terminal. This
is an incompatibility that can't be resolved, so PV can't be used inside <()
in zsh. I'll add this to the known issues list.

However, PV should not be attempting to write to the terminal if you choose
"-q" or "-n", yet it is, so that's a bug in PV, which I'll investigate.

---

Date: Wed, 16 Dec 2015 12:52:41 -0800
From: Frederik
To: Andrew Wood
Subject: Re: problem with <(...) (PV)

By the way, you said earlier that it works in bash, but for my version
of bash (4.3.42), I experience the same problem.

I wonder why you say the problem can't be resolved; consider the
following script:

cat <(for i in $(seq 1 100000); do
        >&2  echo -ne "Processing line $i\\r";
        echo $i;
    done) > /dev/null

This does not hang, and it prints a single line of output which gets
erased and updated...

---

Date: Wed, 16 Dec 2015 21:48:14 +0000
From: Andrew Wood
To: Frederik
Subject: Re: problem with <(...) (PV)

I'm using bash 4.1.2 (CentOS 6).

The problem can't be resolved because PV interacts with the terminal as a
foreground process - it doesn't just write to stderr like your example.

It looks like zsh (and presumably the version of bash you're running) starts
processes inside <() constructs as if they were backgrounded, such that any
attempt to interact with the terminal results in a SIGTTOU being received.

There are probably workarounds that could be done for this, but they would
interfere with another feature of PV, which is that if you start something
like "pv foo > bar", then hit Ctrl-Z and do "bg", then the transfer
continues, but without the progress bar (because PV is now in the
background). You can then do "fg" again, and the progress bar will
re-appear.

[a-j-wood]

Hello

It's been a long time, but I think I have got to the bottom of this. It appears to be down to the fact that processes inside <() are being run in a separate process group, so - because PV is running under a different process group to the process group that owns the terminal - PV thinks it's in the background, so doesn't output anything.

After PV 1.6.20, this will no longer hang PV, and passing "--force" will allow it to output progress.