Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed geodetic tile calculation and cleaned up code. #20

Closed
wants to merge 8 commits into from
Closed

Fixed geodetic tile calculation and cleaned up code. #20

wants to merge 8 commits into from

Conversation

stevendlander
Copy link

The geodetic profile was creating tile caches with a resolution factor of 0.703125, instead of the standard for TMS geodetic tiles which is 1.40625. This error broke tile numbering for subsequent zoom levels.

Additionally, refactored code within the geodetic object for better maintainability.

The geodetic profile was creating tile caches with a resolution factor of 0.703125, instead of the standard for TMS geodetic tiles which is 1.40625.  This error broke tile numbering for subsequent zoom levels.

Additionally, refactored code within the geodetic object for better maintainability.
@rouault
Copy link
Member

rouault commented Jul 10, 2013

It is not obvious to me that this patch is needed.

I have just generated the following :
python swig/python/scripts/gdal2tiles.py -p geodetic world_4326.tif gdal2tiles_output

And when I display gdal2tiles_output/openlayers.html my GeoTIFF is correctly overlayed onto the VMAP layer (after replacing labs.metacarta.com by tilecache.osgeo.org. See http://trac.osgeo.org/gdal/changeset/26162 )

@stevendlander
Copy link
Author

The openlayers geodetic code handles this on line 2207 by subtracting 1 from the server zoom level. This is not necessary, and removing that would make it mirror the mercator code on line 2183.

I would update the pull request but I don't know how, now that it is submitted. :(
EDIT: My ignorance has been overcome. :)

The openlayers generation code subtracted 1 from the server zoom level in order to account for the resolution it was using (0.703125) which should be 1.40625.  This was caused the javascript code to sometimes request TMS tiles with invalid zoom levels (-1) in cases where the map was fully zoomed out.  With this fix, the geodetic openlayers code now more closely follows the same logic as the openlayers code for the mercator profile.
@stevendlander
Copy link
Author

Added a fix for the openlayers browser to the pull request.

A little backstory:

I have been working with MapProxy extensively as a way to turn tiles into sqlite databases. Most of my data is in WGS84 global geodetic format natively, yet in the past I was tiling this data to mercator and then back to WGS84 when creating the sqlite dataset. When I attempted to tile to geodetic directly via gdal2tiles, the result did not match the default profile for global geodetic:

Configuration:
bbox_: [-180.0, -90.0, 180.0, 90.0]
origin_: 'sw'
srs: 'EPSG:4326'
tile_size*: [256, 256]
Levels: Resolutions, # x * y = total tiles
00: 1.40625, # 1 * 1 = 1
01: 0.703125, # 2 * 1 = 2
02: 0.3515625, # 4 * 2 = 8
03: 0.17578125, # 8 * 4 = 32
...
18: 5.364418029785156e-06, # 262144 * 131072 = 34.360G
19: 2.682209014892578e-06, # 524288 * 262144 = 137.439G

Instead, gdal2tiles is calling the level with 2 tiles as level 0. The leaflet group has experienced a similar issue with how MetaCarta handles TMS here: Leaflet/Leaflet#1207 (comment).

You can also see here that OpenLayers make this calculation with 1.40625 being the default value: . While setting the maxResolution field at 0.703125 would indeed fix this issue, I felt it made sense to adhere to a more widely accepted resolution factor.

@rouault
Copy link
Member

rouault commented Jul 11, 2013

Committed in r26164 and r26165

@rouault rouault closed this Jul 11, 2013
@rouault
Copy link
Member

rouault commented Jul 12, 2013

Steven,

there's one aspect that has been overlooked. It is related to generating TMS tiles. What was generated previously was compatible (if one excepts the existing bugs in the x/y inversion in the generated tilemapresource.xml ) of a global-geodetic profile ( http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification#global-geodetic ). This is no longer the case.

@rouault rouault reopened this Jul 12, 2013
@stevendlander
Copy link
Author

Understood, but to my knowledge TMS is not an official OGC standard and thus could have many interpretations. OSGeo has made "level 0 = 0.703125" their interpretation for compatibility with their products.

I was able to find documentation from OGC regarding the successor to TMS, which is WMTS. That specification calls for the resolution change I have outlined here. See page 119 of the PDF located here: http://www.opengeospatial.org/standards/wmts, titled "OpenGIS Web Map Tile Service Implementation Standard."

So will you be keeping the implementation as-is? From the way this discussion has evolved I think either way could be considered correct.

@rouault
Copy link
Member

rouault commented Jul 19, 2013

I'm wondering if there shouldn't be a flag for the user to specifically ask for TMS compatibility, in which case the old behaviour would be kept. Otherwise we would adopt the new one.

@stevendlander
Copy link
Author

I will update code to support such a flag.

@rouault
Copy link
Member

rouault commented Jul 25, 2013

The 2 above commits do not work :

python /home/even/osgeo-gdal-git-full/gdal/swig/python/scripts/gdal2tiles.py -p geodetic -d world_4326.tif world_4326_tms
Traceback (most recent call last):
File "/home/even/osgeo-gdal-git-full/gdal/swig/python/scripts/gdal2tiles.py", line 2277, in
gdal2tiles.process()
File "/home/even/osgeo-gdal-git-full/gdal/swig/python/scripts/gdal2tiles.py", line 481, in process
self.open_input()
File "/home/even/osgeo-gdal-git-full/gdal/swig/python/scripts/gdal2tiles.py", line 994, in open_input
self.geodetic = GlobalGeodetic() # from globalmaptiles.py
File "/home/even/osgeo-gdal-git-full/gdal/swig/python/scripts/gdal2tiles.py", line 358, in init
if self.options.tmscompatible:
AttributeError: 'GlobalGeodetic' object has no attribute 'options'

@stevendlander
Copy link
Author

Yes I am aware, I am not done yet. I was hoping to do it from the git editor but it will take a bit more work on my end. Should be finished up soon.

@stevendlander
Copy link
Author

Should be good to go now.

@rouault
Copy link
Member

rouault commented Jul 25, 2013

Merged 1207814, 9f37a9d and 36af00c in r26204 . I'm confused about b007343 and 2a6b95f ... Are we OK or are there still commits to come ?

@stevendlander
Copy link
Author

All done, thanks for working with me on this. :)

@rouault
Copy link
Member

rouault commented Jul 25, 2013

The code in GlobalGeodetic() is a bit confusing. In the "if tmscompatible is not None:" branch, we use non-TMS resFact, whereas in "if tmscompatible is None" branch we use TMS settings.

@stevendlander
Copy link
Author

I coded the flag so that if the user adds -d, they will get the behavior that the script currently produces. If -d is omitted, then these changes will go in effect.

Would you like that order swapped?

@rouault
Copy link
Member

rouault commented Jul 25, 2013

I'm just saying that the code is confusing. I think the comments are inverted in the GlobalGeodetic() constructor.

That should likely be :
if tmscompatible is not None:
# Defaults the resolution factor to 0.703125 (2 tiles @ level 0)
# Adhers to OSGeo TMS spec http://wiki.osgeo.org/wiki/Tile_Map_Service_Specification#global-geodetic
self.resFact = 180.0 / self.tileSize
else:
# Defaults the resolution factor to 1.40625 (1 tile @ level 0)
# Adheres OpenLayers, MapProxy, etc default resolution for WMTS
self.resFact = 360.0 / self.tileSize

@stevendlander
Copy link
Author

Yes, the comments were backwards. I have been awake far too long. :)

@rouault
Copy link
Member

rouault commented Jul 25, 2013

ok, commited in r26205

@rouault rouault closed this Jul 25, 2013
rouault added a commit that referenced this pull request Mar 8, 2018
This should perhaps help fixing, or at least workarounding, the following
issue seen randomly on the gcc52_stdcpp14_sanitize Travis-CI target.

e.g on https://api.travis-ci.org/v3/job/351015753/log.txt

Running gdrivers/gdalhttp.py...
  TEST: http_1 ... success
ERROR 1: Range downloading not supported by this server!
ERROR 1: Request for 8-407 failed
  TEST: http_2 ... success
  TEST: http_3 ... success
ERROR 4: `/vsicurl/ftp://download.osgeo.org/gdal/data/gtiff/utm.tif' not recognized as a supported file format.
  TEST: http_4 ... HTTP service for ftp://download.osgeo.org/gdal/data/gtiff/utm.tif is down (HTTP Error: ftp error: 425 Security: Bad IP connecting.)
cannot open URL
skip
  TEST: http_5 ... success
ERROR 1: JSON parsing error: unexpected character (at offset 0)
  TEST: http_6 ... success
==31274==WARNING: AddressSanitizer failed to allocate 0x62d00019040f bytes
==31274==AddressSanitizer's allocator is terminating the process instead of returning 0
==31274==If you don't like this behavior set allocator_may_return_null=1
==31274==AddressSanitizer CHECK failed: ../../.././libsanitizer/sanitizer_common/sanitizer_allocator.cc:147 "((0)) != (0)" (0x0, 0x0)
    #0 0x7f3f527259f4  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x9e9f4)
    #1 0x7f3f5272a453 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0xa3453)
    #2 0x7f3f526a7461  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x20461)
    #3 0x7f3f527286d5  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0xa16d5)
    #4 0x7f3f526acb3d  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x25b3d)
    #5 0x7f3f526adbd5  (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x26bd5)
    #6 0x7f3f5271e32e in realloc (/home/travis/build/OSGeo/gdal/install-gcc-5.2.0/lib64/libasan.so.2.0.0+0x9732e)
    #7 0x7f3f3ea2a940 in VSIRealloc /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsisimple.cpp:814
    #8 0x7f3f3e8e3958 in VSICurlHandleWriteFunc /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:839
    #9 0x7f3f2ff61442 in Curl_client_write (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x16442)
    #10 0x7f3f2ff8bc46 in Curl_pp_readresp (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x40c46)
    #11 0x7f3f2ff621ab  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x171ab)
    #12 0x7f3f2ff64546  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x19546)
    #13 0x7f3f2ff61bbf  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x16bbf)
    #14 0x7f3f2ff61cd1  (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x16cd1)
    #15 0x7f3f2ff6776b in Curl_disconnect (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x1c76b)
    #16 0x7f3f2ff7c170 in curl_multi_cleanup (/usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4+0x31170)
    #17 0x7f3f3e902c6b in ClearCache /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:2913
    #18 0x7f3f3e8fc9f7 in ~VSICurlFilesystemHandler /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:2564
    #19 0x7f3f3e8fcf23 in ~VSICurlFilesystemHandler /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil_curl.cpp:2569
    #20 0x7f3f3e9e4a02 in VSIFileManager::~VSIFileManager() /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil.cpp:1805
    #21 0x7f3f3e9e5baa in VSICleanupFileManager /home/travis/build/OSGeo/gdal/gdal/port/cpl_vsil.cpp:1966
    #22 0x7f3f3e533e42 in GDALDriverManager::~GDALDriverManager() /home/travis/build/OSGeo/gdal/gdal/gcore/gdaldrivermanager.cpp:262
    #23 0x7f3f3e53418d in GDALDriverManager::~GDALDriverManager() /home/travis/build/OSGeo/gdal/gdal/gcore/gdaldrivermanager.cpp:329
    #24 0x7f3f3e53a14e in GDALDestroyDriverManager /home/travis/build/OSGeo/gdal/gdal/gcore/gdaldrivermanager.cpp:898
    #25 0x7f3f27794ea3 in ffi_call_unix64 (/usr/lib/python2.7/lib-dynload/_ctypes.so+0x1aea3)
    #26 0x7f3f277948c4 in ffi_call (/usr/lib/python2.7/lib-dynload/_ctypes.so+0x1a8c4)
    #27 0x7f3f277852c1 in _ctypes_callproc (/usr/lib/python2.7/lib-dynload/_ctypes.so+0xb2c1)
    #28 0x7f3f27785aa1  (/usr/lib/python2.7/lib-dynload/_ctypes.so+0xbaa1)
    #29 0x4c2645 in PyObject_Call (/usr/bin/python2.7+0x4c2645)
    #30 0x537589 in PyEval_EvalFrameEx (/usr/bin/python2.7+0x537589)
    #31 0x5376f1 in PyEval_EvalFrameEx (/usr/bin/python2.7+0x5376f1)
    #32 0x5376f1 in PyEval_EvalFrameEx (/usr/bin/python2.7+0x5376f1)
    #33 0x53e2af in PyEval_EvalCodeEx (/usr/bin/python2.7+0x53e2af)
    #34 0x536c45 in PyRun_StringFlags (/usr/bin/python2.7+0x536c45)
    #35 0x53ecb4 in PyRun_SimpleStringFlags (/usr/bin/python2.7+0x53ecb4)
    #36 0x51e62d in Py_Main (/usr/bin/python2.7+0x51e62d)
    #37 0x7f3f504657ec in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x217ec)
    #38 0x41bad0  (/usr/bin/python2.7+0x41bad0)




git-svn-id: https://svn.osgeo.org/gdal/trunk@41669 f0d54148-0727-0410-94bb-9a71ac55c965
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants