Perl enters infinite loop with -d:Trace and binmode(STDERR, encoding(UTF-8))

[p5pRT]

Migrated from rt.perl.org#131115 (status was 'open')

Searchable as RT131115$

[p5pRT]

From @wolfsage

Originally brought to my attention by tudor on #moose​:

These are fine​:

  perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1
  perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' | tee out.txt

These enter infinite loops​:

  perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1 | tee out.txt
  perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1 | grep thing

CPU is at 100%, and if I attach to perl with gdb and grab a backtrace​:

  (gdb) bt
  #0 0x00007fbb8a4a6780 in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.18
  #1 0x00007fbb8a4a761c in PerlIOBuf_write () from /usr/lib/libperl.so.5.18
  #2 0x00007fbb8a4855f5 in Perl_do_print () from /usr/lib/libperl.so.5.18
  #3 0x00007fbb8a40dae7 in Perl_write_to_stderr () from
/usr/lib/libperl.so.5.18
  #4 0x00007fbb8a468c9d in Perl_die_unwind () from /usr/lib/libperl.so.5.18
  #5 0x00007fbb8a40e5a9 in Perl_vcroak () from /usr/lib/libperl.so.5.18
  #6 0x00007fbb8a40efa7 in Perl_croak () from /usr/lib/libperl.so.5.18
  #7 0x00007fbb8a42b6ca in ?? () from /usr/lib/libperl.so.5.18
  #8 0x00007fbb8a433eea in Perl_pp_method () from /usr/lib/libperl.so.5.18
  #9 0x00007fbb8a42b266 in Perl_runops_standard () from
/usr/lib/libperl.so.5.18
  #10 0x00007fbb8a3bc25c in Perl_call_sv () from /usr/lib/libperl.so.5.18
  #11 0x00007fbb88bf4ca0 in PerlIOEncode_flush ()
  from /usr/lib/perl/5.18/auto/PerlIO/encoding/encoding.so
  #12 0x00007fbb8a4a67bf in Perl_PerlIO_flush () from /usr/lib/libperl.so.5.18
  #13 0x00007fbb8a4a761c in PerlIOBuf_write () from /usr/lib/libperl.so.5.18
  #14 0x00007fbb8a4855f5 in Perl_do_print () from /usr/lib/libperl.so.5.18
  #15 0x00007fbb8a42d4cf in Perl_pp_print () from /usr/lib/libperl.so.5.18
  #16 0x00007fbb8a42b266 in Perl_runops_standard () from
/usr/lib/libperl.so.5.18
  #17 0x00007fbb8a3bc25c in Perl_call_sv () from /usr/lib/libperl.so.5.18
#18 0x00007fbb88bf3459 in PerlIOEncode_pushed ()
  from /usr/lib/perl/5.18/auto/PerlIO/encoding/encoding.so
  #19 0x00007fbb8a4a630e in PerlIO_push () from /usr/lib/libperl.so.5.18
  #20 0x00007fbb8a4a6449 in PerlIO_apply_layera () from /usr/lib/libperl.so.5.18
  ---Type <return> to continue, or q <return> to quit---
  #21 0x00007fbb8a4a6528 in PerlIO_apply_layers () from /usr/lib/libperl.so.5.18
  #22 0x00007fbb8a4a6591 in PerlIO_binmode () from /usr/lib/libperl.so.5.18
  #23 0x00007fbb8a472ba0 in Perl_pp_binmode () from /usr/lib/libperl.so.5.18
  #24 0x00007fbb8a42b266 in Perl_runops_standard () from
/usr/lib/libperl.so.5.18
  #25 0x00007fbb8a3c3b44 in perl_run () from /usr/lib/libperl.so.5.18
  #26 0x0000000000400dd9 in main ()

Also tested with 5.24.1, and blead.

-- Matthew Horsfall (alh)

[p5pRT]

From @shlomif

On Fri, 07 Apr 2017 06​:13​:38 -0700
Matthew Horsfall (via RT) <perlbug-followup@​perl.org> wrote​:

Created by Matthew Horsfall
# Please include the string​: [perl #131115]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=131115 >

Originally brought to my attention by tudor on #moose​:

These are fine​:

perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1
perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' | tee out.txt

These enter infinite loops​:

perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1 | tee out.txt
perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1 | grep thing

Just for the record - I can reproduce this problem on Mageia x86-64 v6 (Linux)
with​:

shlomif@​telaviv1​:~$ perl -v

This is perl 5, version 22, subversion 3 (v5.22.3) built for
x86_64-linux-thread-multi (with 1 registered patch, see perl -V for more detail)

Copyright 1987-2017, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http​://www.perl.org/, the Perl Home Page.

shlomif@​telaviv1​:~$

Regards,

  Shlomi Fish

--

---

Shlomi Fish http​://www.shlomifish.org/

How many “one-liners” do I actually write? I don’t know; maybe a couple dozen
a day. But I guess I must be unusual, because as we all know, AWK was a
complete failure and vanished into obscurity since it didn’t address anyone’s
real needs. (That was sarcasm.) — “Twelve Views of Mark Jason Dominus”

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @tonycoz

On Fri, 07 Apr 2017 06​:13​:38 -0700, alh wrote​:

Originally brought to my attention by tudor on #moose​:

These are fine​:

perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1
perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' | tee out.txt

These enter infinite loops​:

perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1 | tee
out.txt
perl -d​:Trace -e 'binmode(STDERR, "​:encoding(UTF-8)");' 2>&1 | grep
thing

CPU is at 100%, and if I attach to perl with gdb and grab a backtrace​:

(gdb) bt
#0 0x00007fbb8a4a6780 in Perl_PerlIO_flush () from
/usr/lib/libperl.so.5.18
#1 0x00007fbb8a4a761c in PerlIOBuf_write () from
/usr/lib/libperl.so.5.18
#2 0x00007fbb8a4855f5 in Perl_do_print () from
/usr/lib/libperl.so.5.18
#3 0x00007fbb8a40dae7 in Perl_write_to_stderr () from
/usr/lib/libperl.so.5.18
#4 0x00007fbb8a468c9d in Perl_die_unwind () from
/usr/lib/libperl.so.5.18
#5 0x00007fbb8a40e5a9 in Perl_vcroak () from /usr/lib/libperl.so.5.18
#6 0x00007fbb8a40efa7 in Perl_croak () from /usr/lib/libperl.so.5.18
#7 0x00007fbb8a42b6ca in ?? () from /usr/lib/libperl.so.5.18
#8 0x00007fbb8a433eea in Perl_pp_method () from
/usr/lib/libperl.so.5.18
#9 0x00007fbb8a42b266 in Perl_runops_standard () from
/usr/lib/libperl.so.5.18
#10 0x00007fbb8a3bc25c in Perl_call_sv () from
/usr/lib/libperl.so.5.18
#11 0x00007fbb88bf4ca0 in PerlIOEncode_flush ()
from /usr/lib/perl/5.18/auto/PerlIO/encoding/encoding.so
#12 0x00007fbb8a4a67bf in Perl_PerlIO_flush () from
/usr/lib/libperl.so.5.18
#13 0x00007fbb8a4a761c in PerlIOBuf_write () from
/usr/lib/libperl.so.5.18
#14 0x00007fbb8a4855f5 in Perl_do_print () from
/usr/lib/libperl.so.5.18
#15 0x00007fbb8a42d4cf in Perl_pp_print () from
/usr/lib/libperl.so.5.18
#16 0x00007fbb8a42b266 in Perl_runops_standard () from
/usr/lib/libperl.so.5.18
#17 0x00007fbb8a3bc25c in Perl_call_sv () from
/usr/lib/libperl.so.5.18
#18 0x00007fbb88bf3459 in PerlIOEncode_pushed ()
from /usr/lib/perl/5.18/auto/PerlIO/encoding/encoding.so
#19 0x00007fbb8a4a630e in PerlIO_push () from /usr/lib/libperl.so.5.18
#20 0x00007fbb8a4a6449 in PerlIO_apply_layera () from
/usr/lib/libperl.so.5.18
---Type <return> to continue, or q <return> to quit---
#21 0x00007fbb8a4a6528 in PerlIO_apply_layers () from
/usr/lib/libperl.so.5.18
#22 0x00007fbb8a4a6591 in PerlIO_binmode () from
/usr/lib/libperl.so.5.18
#23 0x00007fbb8a472ba0 in Perl_pp_binmode () from
/usr/lib/libperl.so.5.18
#24 0x00007fbb8a42b266 in Perl_runops_standard () from
/usr/lib/libperl.so.5.18
#25 0x00007fbb8a3c3b44 in perl_run () from /usr/lib/libperl.so.5.18
#26 0x0000000000400dd9 in main ()

Also tested with 5.24.1, and blead.

I don't have Devel​::Trace installed (I'm working with an uninstalled blead tree), but I see an infinite loop with -Dt with a -DDEBUGGING build too.

I expect what's happening in your case is Devel​::Trace is trying to trace the call to $encoding->renew() or $encoding->needs_lines() (both of which are done with G_EVAL), writing to STDERR, which tries to use the partly initialised PerlIO​::encoding layer and dies, which tries to write to the stream calling PerlIO_flush().

This loops because PerlIO​::encoding prevents recursive calls to $encoding->encode(), losing the output rather than recursving infinitely.

With -Dt (in blead at least) it seems to get stuck earlier in the call to Encode​::find_encoding(), but you get a similar result.

The debugger doesn't seem to have a problem with this code, presumably because it either clones STDERR or opens the TTY.

Possible fixes​:

- Devel​::Trace could clone STDERR and write to that clone, this should work for older perls too.

- perl's -D switch could do something similar.

- perl (perhaps in PerlIO​::encoding) could disable both -Dt tracing and debugger tracing when pushing a layer onto or writing to STDERR. This might be a bit too magical though.

Tony