desalloc problem with GDAL

[laurechr]

the problem present in previous bug 1829 is explain with a litle prog
and with an example

Expected behavior and actual behavior.

For example (please modify !!!): I expected to be able to open this raster file (with a link to
the raster file, or it as an attachment) and it returns an error message
instead.

Steps to reproduce the problem.

the python program is on server bug
this is nsb_test_gdal.py

the raw image is geo_20190827-20190908_sd_2rlks.int is and the associated file is geo_20190827-20190908_sd_2rlks.int.rsc

create a source files directory and put image files
create a destination file directory (easy to delete)
launch the test with this command
nsb_test_gdal.py -sd tgdal -sf geo_20190827-20190908_sd_2rlks.int -dd testima -lo 500

the aim is open a same file and create a "lo" tiff image ( lo is number a file to create)
this test show my problem that the desallocation of dataset (the del command) generate a allocation problem!!
500 or 1000 is a number of images we want create for one run on our studies

this is the result on HPC with 8 cores:
('create fileima :', 'fileima_0')
('create fileima :', 'fileima_1')
('create fileima :', 'fileima_2')
('create fileima :', 'fileima_3')
('create fileima :', 'fileima_4')
('create fileima :', 'fileima_5')
('create fileima :', 'fileima_6')
('create fileima :', 'fileima_7')
('create fileima :', 'fileima_8')
('create fileima :', 'fileima_9')
Segmentation fault

with a a same program in another context there is a segmentaion fault

Dec 10 09:57:05 luke16 kernel: [240697.584561] python[31495]: segfault at
1000000a9 ip 000014ae0c860408 sp 00007fff4348c470 error 4 in
_gdal.so[14ae0c3c6000+1544000]
Dec 10 09:57:05 luke16 kernel: [240697.584852] Code: f8 0f 9c c0 84 c0 0f
84 cd 01 00 00 8b 45 f8 8d 50 01 48 8b 45 d8 89 d6 48 89 c7 e8 b2 b7 f6 ff
48 8b 10 48 81 c2 a8 00 00 00 <48> 8b 12 48 89 c7 ff d2 89 45 e8 83 7d e8
01 74 1b 83 7d e8 00 74

Operating system

Linux luke 3.16.0-5-amd64 #1 SMP Debian 3.16.51-3+deb8u1 (2018-01-08) x86_64
python 2.7

GDAL version and provenance

2.2.4 but i have the problem on the oldest version

[rouault]

Works fine for me with GDAL 2.4 or above. I had to replace the "import nsbas.gdal as gdal" by "from osgeo import gdal". I'd presume something is wrong with your GDAL build or you might run into a bug that has been fixed by a later version.

You might also try running your script under Valgrind: valgrind python .... . Possibly with a debug build of GDAL

[laurechr]

thank's for your reply so i explain the build configuration in first i test my program with a gdal library compiled by the hpc team and shared for the users. i can create 500 image !! i think the configuration of my build is not correct: with our software, a lot of libraries are embedded like gdal we divide the file from compilations in function of type file c fortran perl python so the exe of gdal are put on local bin and the python script are put on local python script we have 2 makefiel can you verify that the compilations rules are good?? these makefile is write by another person an i correct it in function on gdal version thanks in advance for your helps the configuration file define by include ${NSBAS_SOURCE_DIR} /mk/build2.mk contains # Compiler CC = gcc CXX = g++ FC = gfortran # Linker LD = ${CC} # Compilation flags CFLAGS = -O3 -Wall -Wextra -D_BSD_SOURCE -D_FILE_OFFSET_BITS=64 FFLAGS = -cpp -O3 -fcray-pointer -fbackslash -ffixed-line-length-none -ffree-line-length-none -fno-underscoring -D_FILE_OFFSET_BITS=64 # Linkage flags LDFLAGS = OMPFLAGS = -fopenmp -frecursive -lpthread # Specific libs M_LIBS = -lm DL_LIBS = -ldl ICONV_LIBS = RESOLV_LIBS = PTHREAD_LIBS = -lpthread CXX_LIBS = -lstdc++ FORTRAN_LIBS = -lgfortran # Timer used in lapack LAPACK_TIMER = INT_ETIME # Python flags PYTHON_CFLAGS = -fPIC -I/usr/include/python2.7 SBAS_COMMON_SOURCE_DIR= ${NSBAS_SOURCE_DIR} /src/common GDAL_SOURCE_DIR= ${NSBAS_COMMON_SOURCE_DIR} /gdal/build GDAL_BUILD_DIR= ${NSBAS_COMMON_SOURCE_DIR} /gdal/build INCLUDE_SEARCH_FLAGS= \ -I ${NSBAS_SOURCE_DIR} /contrib/fftw/api \ -I ${NSBAS_COMMON_SOURCE_DIR} /base -I ${NSBAS_COMMON_SOURCE_DIR} /blas \ -I ${NSBAS_COMMON_SOURCE_DIR} /math \ -I ${NSBAS_SOURCE_DIR} /include -I. GDAL_CFLAGS = \ -I ${GDAL_BUILD_DIR} /port \ -I ${GDAL_BUILD_DIR} /gcore \ -I ${GDAL_BUILD_DIR} /ogr \ -I ${GDAL_BUILD_DIR} /alg \ -I ${GDAL_BUILD_DIR} /apps CFLAGS+= ${INCLUDE_SEARCH_FLAGS} FFLAGS+= ${INCLUDE_SEARCH_FLAGS} NSBAS_LIBS = ${NSBAS_COMMON_SOURCE_DIR} /math/libnsbas-math.a ${NSBAS_COMMON_SOURCE_DIR} /base/libnsbas-base.a BLAS_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /blas/libnsbas-blas.a FFTW_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /fftw/libnsbas-fftw3f.a HDF5_LIBS= \ ${NSBAS_COMMON_SOURCE_DIR} /hdf5/local/lib/libhdf5_hl.a \ ${NSBAS_COMMON_SOURCE_DIR} /hdf5/local/lib/libhdf5.a LAPACK_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /lapack/libnsbas-lapack.a NETCDF_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /netcdf/libnsbas-netcdf.a GDAL_LIBS= ${NSBAS_SOURCE_DIR} /src/common/gdal/build/libgdal.a ${NSBAS_COMMON_SOURCE_DIR} /hdf5/local/lib/libhdf5.a ${ICONV_LIBS} ${PTHREAD_LIBS} ${DL_LIBS} ${CXX_LIBS} ${M_LIBS} ROIPAC_LIBS= ${NSBAS_COMMON_SOURCE_DIR} /roipac/libnsbas-roipac.a LIBS= ${ROIPAC_LIBS} ${FFTW_LIBS} ${NSBAS_LIBS} ${LAPACK_LIBS} ${BLAS_LIBS} ${M_LIBS} BIN= ${NSBAS_SOURCE_DIR} /bin in attached files this the makefile for gdal exe and for gdal for python binding thanks in advance for your helps christophe laurent De: "Even Rouault" <[email protected]> À: "OSGeo/gdal" <[email protected]> Cc: "CHRISTOPHE LAURENT" <[email protected]>, "Author" <[email protected]> Envoyé: Mercredi 11 Décembre 2019 12:40:34 Objet: Re: [OSGeo/gdal] desalloc problem with GDAL (#2094) Works fine for me with GDAL 2.4 or above. I had to replace the "import nsbas.gdal as gdal" by "from osgeo import gdal". I'd presume something is wrong with your GDAL build or you might run into a bug that has been fixed by a later version. You might also try running your script under Valgrind: valgrind python .... . Possibly with a debug build of GDAL — You are receiving this because you authored the thread. Reply to this email directly, [ #2094?email_source=notifications&email_token=AHIAWJ36W4NLIMOQXZOCRN3QYDGTFA5CNFSM4JZMZUE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGS2EIQ#issuecomment-564503074 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AHIAWJ6RAEI6TVXEM2BUEV3QYDGTFANCNFSM4JZMZUEQ | unsubscribe ] .

[rouault]

with our software, a lot of libraries are embedded like gdal

That might be a problem. You should make sure that you don't have several copies of a same library in different version for example.
Anyway your problem seems to be really linked to your particular setup, and there is no proof that it is something linked to GDAL upstream. That would require in-depth investigation of your setup to figure out the root cause, which is not in the scope of this issue tracker.