Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Source lib/include dirs from gdal_config #37

Closed
wants to merge 1 commit into from
Closed

Source lib/include dirs from gdal_config #37

wants to merge 1 commit into from

Conversation

basilveerman
Copy link

Fixes compilation error when installing python package.

@rouault
Copy link
Member

rouault commented Jul 6, 2014

In which circumstance is it needed ? Which error did you get before ?

@basilveerman
Copy link
Author

The error was incorrect include directories when installing gdal using pip.

This is on a debian wheezy system with libgdal-dev 1.9.0-3.1 installed from debian packages. Use case for not using debian packages for python-gdal is to maintain isolated development environments using virtualenv.

Installing gdal using pip:

$ virtualenv testenv
New python executable in testenv/bin/python
Installing setuptools, pip...done.
$ testenv/bin/pip install numpy -qq
$ testenv/bin/pip install gdal==1.9
Downloading/unpacking gdal==1.9
Downloading GDAL-1.9.0.tar.gz (420kB): 420kB downloaded
Running setup.py (path:/datasets/home/#####/testenv/build/gdal/setup.py) egg_info for package gdal

Installing collected packages: gdal
Running setup.py install for gdal
building 'osgeo._gdal' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/include/python2.7 -I/datasets/home/#####/testenv/local/lib/python2.7/site-packages/numpy/core/include -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
extensions/gdal_wrap.cpp:2853:22: fatal error: cpl_port.h: No such file or directory
#include "cpl_port.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

This is because:

$ gdal-config --prefix
/usr

but:

$ gdal-config --cflags
-I/usr/include/gdal

Results in incorrect include paths as determined in setup.py. While there are many solutions, I thought that using gdal-config to source the directories would avoid the issue entirely.

Is gdal-config not supposed to be relied up on this manner? Output not stable? Would you propose alternative methods to address the problem?

@BishopGIS
Copy link
Member

As I know gdal-config --prefix should return /usr or /usr/local. This is install prefix. So gdal-config output is correct. Also in make file there are to config utilities: gdal-config and gdal-config-inst. The first one points to the source tree and second - to the installed path. It seems to me that during compilation the wrong target executed and the gdal-config (-inst) generated, but should - gdal-config.
This is subject to test under noted os (Debian weesy).
By the way, the current code works on my Ubuntu and I am not sure that proposed changes don't break it.

@basilveerman
Copy link
Author

As I know gdal-config --prefix should return /usr or /usr/local. This is install prefix. So gdal-config output is correct.

Yes, it's correct, but that's not the issue. Getting install prefix and assuming the library/include directories are at [install_prefix]/lib and [install_prefix]/include in setup.py does not seem like good practice. Especially since it's causing errors.

Also in make file there are to config utilities: gdal-config and gdal-config-inst. The first one points to the source tree and second - to the installed path. It seems to me that during compilation the wrong target executed and the gdal-config (-inst) generated, but should - gdal-config.
This is subject to test under noted os (Debian weesy).

I'm not sure I follow. You're saying the the incorrect version of gdal-config is installed on this debian system?

By the way, the current code works on my Ubuntu and I am not sure that proposed changes don't break it.

That's what testing is for. Travis build machines are based on Ubuntu 14.04. If the changes broke the build, issuing python setup.py build in Travis would fail.

Build directly preceding my changes at line 2888:

clang -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/home/travis/build/OSGeo/gdal/gdal/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o

My build at line 2895

clang -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/home/travis/build/OSGeo/gdal/gdal/port -I/home/travis/build/OSGeo/gdal/gdal/gcore -I/home/travis/build/OSGeo/gdal/gdal/alg -I/home/travis/build/OSGeo/gdal/gdal/ogr -I/home/travis/build/OSGeo/gdal/gdal/ogr/ogrsf_frmts -I/home/travis/build/OSGeo/gdal/gdal/frmts/vrt -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o

Removals

-I/home/travis/build/OSGeo/gdal/gdal/include

Additions

-I/home/travis/build/OSGeo/gdal/gdal/port -I/home/travis/build/OSGeo/gdal/gdal/gcore -I/home/travis/build/OSGeo/gdal/gdal/alg -I/home/travis/build/OSGeo/gdal/gdal/ogr -I/home/travis/build/OSGeo/gdal/gdal/ogr/ogrsf_frmts -I/home/travis/build/OSGeo/gdal/gdal/frmts/vrt

This is obviously machine dependent, but it's still finding all relevant system libraries.

This leads me to question: Is this not one of the main purposes of a system config script? To discover details of installed libraries??

@rouault
Copy link
Member

rouault commented Jan 23, 2015

Closing that ticket. I don't want proliferation of issues in GitHub while Trac is still our main ticketing system.

@rouault rouault closed this Jan 23, 2015
rouault added a commit that referenced this pull request Mar 8, 2018
@rouault
/vsicurl/: try to solve random occurrence of invalid use of memory
6ab905d 
This should perhaps help fixing, or at least workarounding, the following
issue seen randomly on the gcc52_stdcpp14_sanitize Travis-CI target.

e.g on https://api.travis-ci.org/v3/job/351015753/log.txt

Running gdrivers/gdalhttp.py...
  TEST: http_1 ... success
ERROR 1: Range downloading not supported by this server!
ERROR 1: Request for 8-407 failed
  TEST: http_2 ... success
  TEST: http_3 ... success
ERROR 4: `/vsicurl/ftp://download.osgeo.org/gdal/data/gtiff/utm.tif' not recognized as a supported file format.
  TEST: http_4 ... HTTP service for ftp://download.osgeo.org/gdal/data/gtiff/utm.tif is down (HTTP Error: ftp error: 425 Security: Bad IP connecting.)
cannot open URL
skip
  TEST: http_5 ... success
ERROR 1: JSON parsing error: unexpected character (at offset 0)
  TEST: http_6 ... success
==31274==WARNING: AddressSanitizer failed to allocate 0x62d00019040f bytes
==31274==AddressSanitizer's allocator is terminating the process instead of returning 0
==31274==If you don't like this behavior set allocator_may_return_null=1
==31274==AddressSanitizer CHECK failed: ../../.././libsanitizer/sanitizer_common/sanitizer_allocator.cc:147 "((0)) != (0)" (0x0, 0x0)
    #0 0x7f3f527259f4  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x9e9f4)
    #1 0x7f3f5272a453 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0xa3453)
    #2 0x7f3f526a7461  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x20461)
    #3 0x7f3f527286d5  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0xa16d5)
    #4 0x7f3f526acb3d  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x25b3d)
    #5 0x7f3f526adbd5  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x26bd5)
    #6 0x7f3f5271e32e in realloc (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x9732e)
    #7 0x7f3f3ea2a940 in VSIRealloc /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsisimple.cpp:814
    #8 0x7f3f3e8e3958 in VSICurlHandleWriteFunc /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:839
    #9 0x7f3f2ff61442 in Curl_client_write (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x16442)
    #10 0x7f3f2ff8bc46 in Curl_pp_readresp (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x40c46)
    #11 0x7f3f2ff621ab  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x171ab)
    #12 0x7f3f2ff64546  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x19546)
    #13 0x7f3f2ff61bbf  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x16bbf)
    #14 0x7f3f2ff61cd1  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x16cd1)
    #15 0x7f3f2ff6776b in Curl_disconnect (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x1c76b)
    #16 0x7f3f2ff7c170 in curl_multi_cleanup (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x31170)
    #17 0x7f3f3e902c6b in ClearCache /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:2913
    #18 0x7f3f3e8fc9f7 in ~VSICurlFilesystemHandler /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:2564
    #19 0x7f3f3e8fcf23 in ~VSICurlFilesystemHandler /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:2569
    #20 0x7f3f3e9e4a02 in VSIFileManager::~VSIFileManager() /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil.cpp:1805
    #21 0x7f3f3e9e5baa in VSICleanupFileManager /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil.cpp:1966
    #22 0x7f3f3e533e42 in GDALDriverManager::~GDALDriverManager() /home/travis/build/OSGeo/gdal/gdal/gcore/gdaldrivermanager.cpp:262
    #23 0x7f3f3e53418d in GDALDriverManager::~GDALDriverManager() /home/travis/build/OSGeo/gdal/gdal/gcore/gdaldrivermanager.cpp:329
    #24 0x7f3f3e53a14e in GDALDestroyDriverManager /home/travis/build/OSGeo/gdal/gdal/gcore/gdaldrivermanager.cpp:898
    #25 0x7f3f27794ea3 in ffi_call_unix64 (/usr/lib/python2.7/lib-dynload/_ctypes.so+0x1aea3)
    #26 0x7f3f277948c4 in ffi_call (/usr/lib/python2.7/lib-dynload/_ctypes.so+0x1a8c4)
    #27 0x7f3f277852c1 in _ctypes_callproc (/usr/lib/python2.7/lib-dynload/_ctypes.so+0xb2c1)
    #28 0x7f3f27785aa1  (/usr/lib/python2.7/lib-dynload/_ctypes.so+0xbaa1)
    #29 0x4c2645 in PyObject_Call (/usr/bin/python2.7+0x4c2645)
    #30 0x537589 in PyEval_EvalFrameEx (/usr/bin/python2.7+0x537589)
    #31 0x5376f1 in PyEval_EvalFrameEx (/usr/bin/python2.7+0x5376f1)
    #32 0x5376f1 in PyEval_EvalFrameEx (/usr/bin/python2.7+0x5376f1)
    #33 0x53e2af in PyEval_EvalCodeEx (/usr/bin/python2.7+0x53e2af)
    #34 0x536c45 in PyRun_StringFlags (/usr/bin/python2.7+0x536c45)
    #35 0x53ecb4 in PyRun_SimpleStringFlags (/usr/bin/python2.7+0x53ecb4)
    #36 0x51e62d in Py_Main (/usr/bin/python2.7+0x51e62d)
    #37 0x7f3f504657ec in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x217ec)
    #38 0x41bad0  (/usr/bin/python2.7+0x41bad0)




git-svn-id: https://svn.osgeo.org/gdal/trunk@41669 f0d54148-0727-0410-94bb-9a71ac55c965
@AsyamMD AsyamMD mentioned this pull request Mar 19, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@basilveerman @rouault @BishopGIS