regex compilation can leak memory

[p5pRT]

Migrated from rt.perl.org#133589 (status was 'open')

Searchable as RT133589$

[p5pRT]

From @khwilliamson

This is a bug report for perl from khw@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.29.4.

---

In looking at the code in regcomp.c, I noticed that one way it tries to
avoid leaks when raising errors is that it mortalizes an SV before
dying. This works, but the scalar doesn't get mortalized if warnings
are generated, so if those are fatalized there would be a leak. Further,
other functions are called which can output warnings or errors, and
those functions don't mortalize the SV. As an example, grok_hex() can
be called with options that allow it to output warnings. If those are
fatal, a leak will occur. User-defined \p{} properties call perl code
which can die or warn; again leaks can occur.

---

---

Flags​:
  category=core
  severity=low

---

Site configuration information for perl 5.29.4​:

Configured by khw at Sun Oct 14 10​:24​:54 MDT 2018.

Summary of my perl5 (revision 5 version 29 subversion 4) configuration​:
  Commit id​: 7bf15fe
  Platform​:
  osname=linux
  osvers=4.15.0-36-generic
  archname=x86_64-linux-thread-multi-ld
  uname='linux khw-xps-8930 4.15.0-36-generic #39-ubuntu smp mon sep
24 16​:19​:09 utc 2018 x86_64 x86_64 x86_64 gnulinux '
  config_args='-des -Uversiononly -Dprefix=/home/khw/blead -Dusedevel
-A'optimize=-ggdb3' -A'optimize=-O0' -Accflags='-Wno-deprecated'
-Accflags='-DPERL_EXTERNAL_GLOB' -Dman1dir=none -Dman3dir=none -Dcc=g++
-DDEBUGGING -Dusemorebits -Dusecbacktrace -Dusethreads'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=define
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='g++'
  ccflags ='-D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -DUSE_C_BACKTRACE -g
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O2 -ggdb3 -O0'
  cppflags='-D_REENTRANT -D_GNU_SOURCE -Wno-deprecated
-DPERL_EXTERNAL_GLOB -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='7.3.0'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='long double'
  nvsize=16
  Off_t='off_t'
  lseeksize=8
  alignbytes=16
  prototype=define
  Linker and Libraries​:
  ld='g++'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/include/c++/7 /usr/include/x86_64-linux-gnu/c++/7
/usr/include/c++/7/backward /usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib
/usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.27.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.27'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -ggdb3 -O0 -L/usr/local/lib
-fstack-protector-strong'

---

@​INC for perl 5.29.4​:
  /home/khw/perl/blead/lib
  /home/khw/perl/blead/t
  /home/khw/blead/lib/perl5/site_perl/5.29.4/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/site_perl/5.29.4
  /home/khw/blead/lib/perl5/5.29.4/x86_64-linux-thread-multi-ld
  /home/khw/blead/lib/perl5/5.29.4
  /home/khw/blead/lib/perl5/site_perl/5.29.3
  /home/khw/blead/lib/perl5/site_perl/5.29.2
  /home/khw/blead/lib/perl5/site_perl/5.29.1
  /home/khw/blead/lib/perl5/site_perl/5.29.0
  /home/khw/blead/lib/perl5/site_perl/5.28.0
  /home/khw/blead/lib/perl5/site_perl/5.27.11
  /home/khw/blead/lib/perl5/site_perl/5.27.10
  /home/khw/blead/lib/perl5/site_perl/5.27.9
  /home/khw/blead/lib/perl5/site_perl/5.27.8
  /home/khw/blead/lib/perl5/site_perl/5.27.7
  /home/khw/blead/lib/perl5/site_perl/5.27.6
  /home/khw/blead/lib/perl5/site_perl/5.27.5
  /home/khw/blead/lib/perl5/site_perl/5.27.4
  /home/khw/blead/lib/perl5/site_perl/5.27.3
  /home/khw/blead/lib/perl5/site_perl/5.27.2
  /home/khw/blead/lib/perl5/site_perl/5.27.1
  /home/khw/blead/lib/perl5/site_perl/5.27.0
  /home/khw/blead/lib/perl5/site_perl/5.26.0
  /home/khw/blead/lib/perl5/site_perl/5.25.12
  /home/khw/blead/lib/perl5/site_perl/5.25.11
  /home/khw/blead/lib/perl5/site_perl/5.25.10
  /home/khw/blead/lib/perl5/site_perl/5.25.9
  /home/khw/blead/lib/perl5/site_perl/5.25.8
  /home/khw/blead/lib/perl5/site_perl/5.25.7
  /home/khw/blead/lib/perl5/site_perl/5.25.6
  /home/khw/blead/lib/perl5/site_perl/5.25.5
  /home/khw/blead/lib/perl5/site_perl/5.25.4
  /home/khw/blead/lib/perl5/site_perl/5.25.3
  /home/khw/blead/lib/perl5/site_perl/5.25.2
  /home/khw/blead/lib/perl5/site_perl/5.25.1
  /home/khw/blead/lib/perl5/site_perl/5.24.0
  /home/khw/blead/lib/perl5/site_perl/5.23.10
  /home/khw/blead/lib/perl5/site_perl/5.23.9
  /home/khw/blead/lib/perl5/site_perl/5.23.8
  /home/khw/blead/lib/perl5/site_perl/5.23.7
  /home/khw/blead/lib/perl5/site_perl/5.23.6
  /home/khw/blead/lib/perl5/site_perl/5.23.5
  /home/khw/blead/lib/perl5/site_perl/5.23.4
  /home/khw/blead/lib/perl5/site_perl/5.23.3
  /home/khw/blead/lib/perl5/site_perl/5.23.2
  /home/khw/blead/lib/perl5/site_perl/5.23.1
  /home/khw/blead/lib/perl5/site_perl/5.23.0
  /home/khw/blead/lib/perl5/site_perl/5.22.0
  /home/khw/blead/lib/perl5/site_perl/5.21.12
  /home/khw/blead/lib/perl5/site_perl/5.21.11
  /home/khw/blead/lib/perl5/site_perl/5.21.10
  /home/khw/blead/lib/perl5/site_perl/5.21.9
  /home/khw/blead/lib/perl5/site_perl/5.21.8
  /home/khw/blead/lib/perl5/site_perl/5.21.7
  /home/khw/blead/lib/perl5/site_perl/5.21.6
  /home/khw/blead/lib/perl5/site_perl/5.21.5
  /home/khw/blead/lib/perl5/site_perl/5.21.4
  /home/khw/blead/lib/perl5/site_perl/5.21.3
  /home/khw/blead/lib/perl5/site_perl/5.21.2
  /home/khw/blead/lib/perl5/site_perl/5.21.1
  /home/khw/blead/lib/perl5/site_perl/5.20.0
  /home/khw/blead/lib/perl5/site_perl/5.19.12
  /home/khw/blead/lib/perl5/site_perl/5.19.11
  /home/khw/blead/lib/perl5/site_perl/5.19.10
  /home/khw/blead/lib/perl5/site_perl

---

Environment for perl 5.29.4​:
  HOME=/home/khw
  LANG=en_US.UTF-8
  LANGUAGE=en_US
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)

PATH=/usr/lib/ccache​:/home/khw/bin​:/home/khw/perl5/perlbrew/bin​:/home/khw/print/bin​:/bin​:/usr/local/sbin​:/usr/local/bin​:/usr/sbin​:/usr/bin​:/sbin​:/usr/games​:/usr/local/games​:/snap/bin​:/home/khw/iands/www​:/home/khw/cxoffice/bin
  PERL5OPT=-w
  PERL_BADLANG (unset)
  PERL_DIFF_TOOL=wgdiff
  PERL_POD_PEDANTIC=1
  SHELL=/bin/ksh

[p5pRT]

From @jkeenan

On Sun, 14 Oct 2018 18​:56​:50 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.29.4.

-----------------------------------------------------------------
In looking at the code in regcomp.c, I noticed that one way it tries
to
avoid leaks when raising errors is that it mortalizes an SV before
dying. This works, but the scalar doesn't get mortalized if warnings
are generated, so if those are fatalized there would be a leak.
Further,
other functions are called which can output warnings or errors, and
those functions don't mortalize the SV. As an example, grok_hex() can
be called with options that allow it to output warnings. If those are
fatal, a leak will occur. User-defined \p{} properties call perl code
which can die or warn; again leaks can occur.

Could a TODO test be written for this?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @khwilliamson

On 12/6/18 1​:26 PM, James E Keenan via RT wrote​:

On Sun, 14 Oct 2018 18​:56​:50 GMT, public@​khwilliamson.com wrote​:

This is a bug report for perl from khw@​cpan.org,
generated with the help of perlbug 1.41 running under perl 5.29.4.

-----------------------------------------------------------------
In looking at the code in regcomp.c, I noticed that one way it tries
to
avoid leaks when raising errors is that it mortalizes an SV before
dying. This works, but the scalar doesn't get mortalized if warnings
are generated, so if those are fatalized there would be a leak.
Further,
other functions are called which can output warnings or errors, and
those functions don't mortalize the SV. As an example, grok_hex() can
be called with options that allow it to output warnings. If those are
fatal, a leak will occur. User-defined \p{} properties call perl code
which can die or warn; again leaks can occur.

Could a TODO test be written for this?

Thank you very much.

Certainly. I know of various cases where it happens, but it would be
best to fix it in the general case, and I don't know all the places it
could happen.