OSX: fix build failures due to json-c

[alanstewart-terragotech]

What does this PR do?

Resolve 64-bit build failures on OSX 10.12.6., following .travis.yml and using gdal/ci/travis/osx/*.sh. Updates assignment of the return value from json_object_array_lenght() to type 'auto' instead of 'int'. Some issues resolved by applying static_cast().

What are related issues/pull requests?

None.

Tasklist

• ADD YOUR TASKS HERE
• Add test case(s)
• Review
• Adjust for comments
• All CI builds and checks have passed

Environment

OSX 10.12.6, xcode, homebrew

[rouault]

unrelated change. breaks some build like the ubuntu_1804 of https://travis-ci.org/OSGeo/gdal/builds/476518997

[alanstewart-terragotech]

Fails locally if it’s not there. Alan Stewart Senior Software Engineer TerraGo Technologies 3200 Windy Hill Road, Suite 1550W Atlanta, GA 30339 USA O. +1 678.391.9615 www.terragotech.com<applewebdata://B24C0762-C7C9-4431-8518-ACC915448B89/www.terragotech.com> From: Even Rouault <[email protected]> Sent: Monday, January 07, 2019 3:36 PM To: OSGeo/gdal <[email protected]> Cc: Alan Stewart <[email protected]>; Author <[email protected]> Subject: Re: [OSGeo/gdal] OSX: fix build failures due to json-c (#1179) @rouault commented on this pull request.

________________________________ In gdal/ogr/ogrsf_frmts/sqlite/ogrsqlitevirtualogr.cpp<#1179 (comment)>:

@@ -1713,6 +1713,7 @@ static const struct sqlite3_module sOGR2SQLITEModule =

nullptr, // xSavepoint nullptr, // xRelease nullptr, // xRollbackTo + nullptr, // xShadowName unrelated change. breaks some build like the ubuntu_1804 of https://travis-ci.org/OSGeo/gdal/builds/476518997 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1179 (review)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AG_c2tnTeIwcgk9X9Hh590huV-wKRfIkks5vA6-5gaJpZM4Z0JFJ>.

[alanstewart-terragotech]

Appears to have been added to sqlite very recently, 2018—11-05, if I read their file history correctly. Alan Stewart Senior Software Engineer TerraGo Technologies 3200 Windy Hill Road, Suite 1550W Atlanta, GA 30339 USA O. +1 678.391.9615 www.terragotech.com<applewebdata://B24C0762-C7C9-4431-8518-ACC915448B89/www.terragotech.com> From: Even Rouault <[email protected]> Sent: Monday, January 07, 2019 3:36 PM To: OSGeo/gdal <[email protected]> Cc: Alan Stewart <[email protected]>; Author <[email protected]> Subject: Re: [OSGeo/gdal] OSX: fix build failures due to json-c (#1179) @rouault commented on this pull request.

________________________________ In gdal/ogr/ogrsf_frmts/sqlite/ogrsqlitevirtualogr.cpp<#1179 (comment)>:

@@ -1713,6 +1713,7 @@ static const struct sqlite3_module sOGR2SQLITEModule =

nullptr, // xSavepoint nullptr, // xRelease nullptr, // xRollbackTo + nullptr, // xShadowName unrelated change. breaks some build like the ubuntu_1804 of https://travis-ci.org/OSGeo/gdal/builds/476518997 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1179 (review)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AG_c2tnTeIwcgk9X9Hh590huV-wKRfIkks5vA6-5gaJpZM4Z0JFJ>.

[rouault]

Fails locally if it’s not there.

you probably need to add a check for a different sqlite version
#if SQLITE_VERSION_NUMBER >= version_where_xShadow_was_first_introduced
Another option, much more future proof, would probably be to avoid declaring a static const struct sqlite3_module and just declare a local variable in OGR2SQLITEModule::Setup() where it is used, and just manually set the members of version 1, that is all except the ones starting at #if SQLITE_VERSION_NUMBER >= 3007007L.

struct sqlite3_module sModule;
sModule.iVersion = 1;
sModule.xCreate = OGR2SQLITE_ConnectCreate;
etc...

[alanstewart-terragotech]

Sqlite 3.25.3 seems to be the magic number. All green now except for ubuntu_1804, which has pytest failures in in the HDF5 format. I think the only changes to bagdataset.cpp are static_cast<int>()’s. What would be different on this platform versus all the others? ==================================== ERRORS ====================================

_________________ ERROR at teardown of test_bag_write_south_up _________________ @pytest.fixture(autouse=True, scope='module') def check_no_file_leaks(): num_files = len(gdaltest.get_opened_files()) yield diff = len(gdaltest.get_opened_files()) - num_files

assert diff == 0, 'Leak of file handles: %d leaked' % diff

E AssertionError: Leak of file handles: 1 leaked E assert 1 == 0 gdrivers/bag.py:51: AssertionError ----------------------------- Captured stderr call ----------------------------- HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5Adeprec.c line 130 in H5Acreate1(): not a location major: Invalid arguments to routine minor: Inappropriate type #1: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID major: Invalid arguments to routine minor: Bad value HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5G.c line 725 in H5Gclose(): not a group major: Invalid arguments to routine minor: Inappropriate type =================================== FAILURES ===================================

__________________________ test_bag_write_single_band __________________________ def test_bag_write_single_band(): if gdaltest.bag_drv is None: pytest.skip() tst = gdaltest.GDALTest('BAG', 'byte.tif', 1, 4672) ret = tst.testCreateCopy(quiet_error_handler=False,

new_filename='/vsimem/out.bag')

bag.py:614: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <gdaltest.GDALTest object at 0x7f0fcfaba550>, check_minmax = 1 check_gt = 0, check_srs = None, vsimem = 0, new_filename = '/vsimem/out.bag' strict_in = 0, skip_preclose_test = 0, delete_copy = 1, gt_epsilon = None check_checksum_not_null = None, interrupt_during_copy = False dest_open_options = None, quiet_error_handler = False def testCreateCopy(self, check_minmax=1, check_gt=0, check_srs=None, vsimem=0, new_filename=None, strict_in=0, skip_preclose_test=0, delete_copy=1, gt_epsilon=None, check_checksum_not_null=None, interrupt_during_copy=False, dest_open_options=None, quiet_error_handler=True): self.testDriver() if self.filename_absolute: wrk_filename = self.filename else: wrk_filename = 'data/' + self.filename if self.open_options: src_ds = gdal.OpenEx(wrk_filename, gdal.OF_RASTER, open_options=self.open_options) else: src_ds = gdal.Open(wrk_filename, gdal.GA_ReadOnly) if self.band > 0: minmax = src_ds.GetRasterBand(self.band).ComputeRasterMinMax() src_prj = src_ds.GetProjection() src_gt = src_ds.GetGeoTransform() if new_filename is None: if vsimem: new_filename = '/vsimem/' + os.path.basename(self.filename) + '.tst' else: new_filename = 'tmp/' + os.path.basename(self.filename) + '.tst' if quiet_error_handler: gdal.PushErrorHandler('CPLQuietErrorHandler') if interrupt_during_copy: new_ds = self.driver.CreateCopy(new_filename, src_ds, strict=strict_in, options=self.options, callback=testCreateCopyInterruptCallback) else: new_ds = self.driver.CreateCopy(new_filename, src_ds, strict=strict_in, options=self.options) if quiet_error_handler: gdal.PopErrorHandler() if interrupt_during_copy: if new_ds is None: gdal.PushErrorHandler('CPLQuietErrorHandler') self.driver.Delete(new_filename) gdal.PopErrorHandler() return new_ds = None self.driver.Delete(new_filename) pytest.fail('CreateCopy() should have failed due to interruption')

assert new_ds is not None, ('Failed to create test file using CreateCopy method.' +

'\n' + gdal.GetLastErrorMsg()) E AssertionError: Failed to create test file using CreateCopy method. E E assert None is not None ../pymod/gdaltest.py:405: AssertionError ----------------------------- Captured stderr call ----------------------------- HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5Adeprec.c line 130 in H5Acreate1(): not a location major: Invalid arguments to routine minor: Inappropriate type #1: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID major: Invalid arguments to routine minor: Bad value HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5G.c line 725 in H5Gclose(): not a group major: Invalid arguments to routine minor: Inappropriate type

___________________________ test_bag_write_two_bands ___________________________ def test_bag_write_two_bands(): if gdaltest.bag_drv is None: pytest.skip() tst = gdaltest.GDALTest('BAG', 'test_vr.bag', 2, 60, options=['BLOCK_SIZE=2', 'VAR_ABSTRACT=foo', 'VAR_XML_IDENTIFICATION_CITATION=<bar/>']) tst.testCreateCopy(quiet_error_handler=False, delete_copy=False,

new_filename='/vsimem/out.bag')

bag.py:632: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <gdaltest.GDALTest object at 0x7f0fca64e990>, check_minmax = 1 check_gt = 0, check_srs = None, vsimem = 0, new_filename = '/vsimem/out.bag' strict_in = 0, skip_preclose_test = 0, delete_copy = False, gt_epsilon = None check_checksum_not_null = None, interrupt_during_copy = False dest_open_options = None, quiet_error_handler = False def testCreateCopy(self, check_minmax=1, check_gt=0, check_srs=None, vsimem=0, new_filename=None, strict_in=0, skip_preclose_test=0, delete_copy=1, gt_epsilon=None, check_checksum_not_null=None, interrupt_during_copy=False, dest_open_options=None, quiet_error_handler=True): self.testDriver() if self.filename_absolute: wrk_filename = self.filename else: wrk_filename = 'data/' + self.filename if self.open_options: src_ds = gdal.OpenEx(wrk_filename, gdal.OF_RASTER, open_options=self.open_options) else: src_ds = gdal.Open(wrk_filename, gdal.GA_ReadOnly) if self.band > 0: minmax = src_ds.GetRasterBand(self.band).ComputeRasterMinMax() src_prj = src_ds.GetProjection() src_gt = src_ds.GetGeoTransform() if new_filename is None: if vsimem: new_filename = '/vsimem/' + os.path.basename(self.filename) + '.tst' else: new_filename = 'tmp/' + os.path.basename(self.filename) + '.tst' if quiet_error_handler: gdal.PushErrorHandler('CPLQuietErrorHandler') if interrupt_during_copy: new_ds = self.driver.CreateCopy(new_filename, src_ds, strict=strict_in, options=self.options, callback=testCreateCopyInterruptCallback) else: new_ds = self.driver.CreateCopy(new_filename, src_ds, strict=strict_in, options=self.options) if quiet_error_handler: gdal.PopErrorHandler() if interrupt_during_copy: if new_ds is None: gdal.PushErrorHandler('CPLQuietErrorHandler') self.driver.Delete(new_filename) gdal.PopErrorHandler() return new_ds = None self.driver.Delete(new_filename) pytest.fail('CreateCopy() should have failed due to interruption')

assert new_ds is not None, ('Failed to create test file using CreateCopy method.' +

'\n' + gdal.GetLastErrorMsg()) E AssertionError: Failed to create test file using CreateCopy method. E E assert None is not None ../pymod/gdaltest.py:405: AssertionError ----------------------------- Captured stderr call ----------------------------- HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5Adeprec.c line 130 in H5Acreate1(): not a location major: Invalid arguments to routine minor: Inappropriate type #1: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID major: Invalid arguments to routine minor: Bad value HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5G.c line 725 in H5Gclose(): not a group major: Invalid arguments to routine minor: Inappropriate type

___________________________ test_bag_write_south_up ____________________________ def test_bag_write_south_up(): if gdaltest.bag_drv is None: pytest.skip() # Generate a south-up dataset src_ds = gdal.Warp('', 'data/byte.tif', format='MEM', outputBounds=[440720, 3751320, 441920, 3750120]) # Translate it into BAG ds = gdal.Translate('/vsimem/out.bag', src_ds) # Check that it is presented as north-up

cs = ds.GetRasterBand(1).Checksum()

E AttributeError: 'NoneType' object has no attribute 'GetRasterBand' bag.py:659: AttributeError ----------------------------- Captured stderr call ----------------------------- HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5Adeprec.c line 130 in H5Acreate1(): not a location major: Invalid arguments to routine minor: Inappropriate type #1: ../../../src/H5Gloc.c line 253 in H5G_loc(): invalid object ID major: Invalid arguments to routine minor: Bad value HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 139706346579776: #000: ../../../src/H5G.c line 725 in H5Gclose(): not a group major: Invalid arguments to routine minor: Inappropriate type Alan Stewart Senior Software Engineer TerraGo Technologies 3200 Windy Hill Road, Suite 1550W Atlanta, GA 30339 USA O. +1 678.391.9615 www.terragotech.com<applewebdata://B24C0762-C7C9-4431-8518-ACC915448B89/www.terragotech.com> From: Even Rouault <[email protected]> Sent: Monday, January 07, 2019 3:53 PM To: OSGeo/gdal <[email protected]> Cc: Alan Stewart <[email protected]>; Author <[email protected]> Subject: Re: [OSGeo/gdal] OSX: fix build failures due to json-c (#1179) Fails locally if it’s not there. you probably need to add a check for a different sqlite version #if SQLITE_VERSION_NUMBER >= version_where_xShadow_was_first_introduced Another option, much more future proof, would probably be to avoid declaring a static const struct sqlite3_module and just declare a local variable in OGR2SQLITEModule::Setup() where it is used, and just manually set the members of version 1, that is all except the ones starting at #if SQLITE_VERSION_NUMBER >= 3007007L. struct sqlite3_module sModule; sModule.iVersion = 1; sModule.xCreate = OGR2SQLITE_ConnectCreate; etc... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1179 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AG_c2gnNTXIHIvPm5bHC2_k_9nZOmsNZks5vA7OrgaJpZM4Z0JFJ>.

[rouault]

Your changes in bagdataset.cpp are not appropriate and should be reverted. Since libhdf5 1.10, hid_t is a typedef of int64 whereas it was int in previous versions, hence only the failure on the most recent distribution

I believe that it can be fixed with changing h5check() declaration to be a template
Something like

static template<class T> T h5check(T ret, const char* filename, int line)

[alanstewart-terragotech]

The only failure now, gcc52_stdcpp14_sanitize, appears to be a network connectivity issue. How to force a re-run?

[rouault]

The only failure now, gcc52_stdcpp14_sanitize, appears to be a network connectivity issue. How to force a re-run?

I've triggered a re-run. You probably can't because of missing rights on the gdal repository

[alanstewart-terragotech]

All green now!