Skip to content

Commit

Permalink
Support QGIS Datasource on Map Composition
Browse files Browse the repository at this point in the history 
- This will enable inserting Tile URL as basemap stored in QGIS Project
- Update README
  • Loading branch information
@lucernae
lucernae committed Aug 3, 2018
1 parent f9faedc commit ddf4b75
Show file tree
Hide file tree
Showing 3 changed files with 259 additions and 53 deletions.
50 changes: 46 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Create a QGIS project on the fly on the server

* Supported raster formats : asc, tif, tiff
* Supported raster formats : asc, tif, tiff, geotiff, geotif
* Supported vector formats : shp, geojson

**This project is still in development, the API may change.**
Expand All @@ -10,11 +10,12 @@
* Parameters :
* SERVICE=MAPCOMPOSITION, compulsory
* PROJECT, compulsory, path where the project will be written on the file system.
* FILES, compulsory, it's a list of files on the filesystem, separated by a semicolon.
* SOURCES, compulsory, it's a list of layer sources. It can be tile url or QGIS DataSource URI or files on the filesystem, separated by a semicolon.
Especially for QGIS DataSource URI, it must be url quoted twice (first the url, second the whole datasource string).
* FILES, optional, legacy parameter, it's a list of files on the filesystem, separated by a semicolon. It is overriden by SOURCES.
* NAMES, compulsory, it's a list of names, separated by a semicolon. It will be used for the legend. Items in this list should match layers in the FILES list.
* OVERWRITE, optional, false by default. Boolean if we can overwrite the existing PROJECT above. Values can be '1', 'YES', 'TRUE', 'yes', 'true'.
* REMOVEQML, optional, false by default. Boolean if we can remove the QML. The style is already in the QGS file. Values can be '1', 'YES', 'TRUE', 'yes', 'true'.
* BASEMAP, optional, None by default. it's a list of string comprised of a tile url and its service name, separated by a semicolon.


Layers need to be stored on the server's filesystem. The project will be created at the specified path above, on the server's filesystem too.
Expand All @@ -40,6 +41,47 @@ MAP=/destination/project.qgs&
REQUEST=GetCapabilities
```

* Example of using SOURCES with tile URI:

```
http://localhost:81/qgis?
SERVICE=MAPCOMPOSITION&
PROJECT=/destination/project.qgs&
SOURCES=type%253Dxyz%2526url%253Dhttp%25253A%2F%2Fa.tile.osm.org%2F%25257Bz%25257D%2F%25257Bx%25257D%2F%25257By%25257D.png;/path/1.shp;/path/2.geojson;/path/3.asc&
NAMES=Basemap;My layer 1;MyLayer 2;Layer 3&
OVERWRITE=true
```

In the sample request above, note that the datasource: ```type%253Dxyz%2526url%253Dhttp%25253A%2F%2Fa.tile.osm.org%2F%25257Bz%25257D%2F%25257Bx%25257D%2F%25257By%25257D.png``` were urlquoted twice.
The actual datasource is: ```type=xyz&url=http%3A//a.tile.osm.org/%7Bz%7D/%7Bx%7D/%7By%7D.png```
Note that the actual url is: ```http://a.tile.osm.org/{z}/{x}/{y}.png```

Thus in order to send the request, the url needs to be quoted first before it was inserted into datasource uri (to quote & symbol and = from url).
Then, the datasource needs to be quoted again, because it was sent via GET requests url (to quote & symbol and = from datasource query params).

As an example of sending your datasource of base layer:

Quote your tile url:

```
>>> tile_url = 'http://a.tile.osm.org/{z}/{x}/{y}.png'
>>> from requests.utils import quote
>>> tile_url = quote(tile_url)
```

Then build your data source definition and quote it:

```
>>> definition = {
... 'url': tile_url,
... 'type': 'xyz'
... }
>>> datasource = '&'.join(['{key}={value}'.format(key=key,value=value) for key,value in definition.iteritems()])
'url=http%3A//a.tile.osm.org/%7Bz%7D/%7Bx%7D/%7By%7D.png&type=xyz'
>>> quoted_datasource = quote(datasource)
'url%3Dhttp%253A//a.tile.osm.org/%257Bz%257D/%257Bx%257D/%257By%257D.png%26type%3Dxyz'
```

## Todo
* Add WCS
* Add tests
Expand Down Expand Up @@ -107,4 +149,4 @@ LAYERS=your_quoted_layers_variable
```
docker pull kartoza/qgis-server:LTR
QUERY_STRING="SERVICE=STYLEMANAGER&PROJECT=/usr/src/app/geonode/qgis_layer/small_building.qgs&REQUEST=GetStyle&LAYER=build&NAME=toto" /usr/lib/cgi-bin/qgis_mapserv.fcgi
```
```
149 changes: 100 additions & 49 deletions filters/map_composition.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,12 @@
QgsMessageLog,
QgsVectorLayer,
QgsRasterLayer)
from .tools import generate_legend

from .tools import (
generate_legend,
validate_source_uri,
is_file_path,
layer_from_source)


class MapComposition(QgsServerFilter):
Expand All @@ -43,11 +48,12 @@ def responseComplete(self):
Example :
SERVICE=MAPCOMPOSITION&
PROJECT=/destination/project.qgs&
FILES=/path/1.shp;/path/2.shp;/path/3.asc&
SOURCES=type=xyz&url=http://tile.osm.org/{z}/{x}/{y}.png?layers=osm;
/path/1.shp;/path/2.shp;/path/3.asc&
FILES={Legacy Name for Sources Parameter}
NAMES=Layer 1;Layer 2;Layer 3&
REMOVEQML=true&
OVERWRITE=true&
BASEMAP='type=xyz&url=http://tile.osm.org/{z}/{x}/{y}.png?layers=osm;osm'
"""
request = self.serverInterface().requestHandler()
params = request.parameterMap()
Expand Down Expand Up @@ -80,94 +86,139 @@ def responseComplete(self):
else:
remove_qml = False

if exists(project_path):
if not overwrite:
msg = 'PROJECT is already existing : %s \n' % project_path
request.appendBody(msg)
return
else:
remove(project_path)
if exists(project_path) and overwrite:
# Overwrite means create from scratch again
remove(project_path)

files_parameters = params.get('FILES')
if not files_parameters:
request.appendBody('FILES parameter is missing.\n')
sources_parameters = params.get('SOURCES')
# support legacy params: FILES
if not sources_parameters:
sources_parameters = params.get('FILES')

if not sources_parameters:
request.appendBody('SOURCES parameter is missing.\n')
return

files = files_parameters.split(';')
for layer_file in files:
if not exists(layer_file):
request.appendBody('file not found : %s.\n' % layer_file)
sources = sources_parameters.split(';')
for layer_source in sources:

if not validate_source_uri(layer_source):
request.appendBody(
'invalid parameter: {0}.\n'.format(layer_source))
return

if is_file_path(layer_source):
if not exists(layer_source):
request.appendBody('file not found : %s.\n' % layer_source)
return

names_parameters = params.get('NAMES', None)
if names_parameters:
names = names_parameters.split(';')
if len(names) != len(files):
if len(names) != len(sources):
request.appendBody(
'Not same length between NAMES and FILES')
'Not same length between NAMES and SOURCES')
return
else:
names = [
splitext(basename(layer_file))[0] for layer_file in files]
splitext(basename(layer_source))[0]
for layer_source in sources]

QgsMessageLog.logMessage('Setting up project to %s' % project_path)
project = QgsProject.instance()
project.setFileName(project_path)
if exists(project_path) and not overwrite:
project.read()

qml_files = []
qgis_layers = []
vector_layers = []
raster_layer = []
raster_layers = []

for layer_name, layer_file in zip(names, files):
if layer_file.endswith(('shp', 'geojson')):
qgis_layer = QgsVectorLayer(layer_file, layer_name, 'ogr')
vector_layers.append(qgis_layer.id())
for layer_name, layer_source in zip(names, sources):

elif layer_file.endswith(('asc', 'tiff', 'tif')):
qgis_layer = QgsRasterLayer(layer_file, layer_name)
raster_layer.append(qgis_layer.id())
else:
request.appendBody('Invalid format : %s' % layer_file)
qgis_layer = layer_from_source(layer_source, layer_name)

if not qgis_layer:
request.appendBody('Invalid format : %s' % layer_source)
return

if not qgis_layer.isValid():
request.appendBody('Layer is not valid : %s' % layer_file)
request.appendBody('Layer is not valid : %s' % layer_source)
return

if isinstance(qgis_layer, QgsRasterLayer):
raster_layers.append(qgis_layer.id())
elif isinstance(qgis_layer, QgsVectorLayer):
vector_layers.append(qgis_layer.id())
else:
request.appendBody('Invalid type : {0} - {1}'.format(
qgis_layer, type(qgis_layer)))

qgis_layers.append(qgis_layer)

qml_file = splitext(layer_file)[0] + '.qml'
qml_file = splitext(layer_source)[0] + '.qml'
if exists(qml_file):
# Check if there is a QML
qml_files.append(qml_file)

style_manager = qgis_layer.styleManager()
style_manager.renameStyle('', 'default')

# add basemap to the qgs project for a background on a thumbnail
if params.get('BASEMAP'):
# basemap is comprised of url and name with semicolon as separator
basemap = params.get('BASEMAP').split(';')
if len(basemap) == 2:
qgis_layer = QgsRasterLayer(basemap[0], basemap[1], 'wms')
if not qgis_layer.isValid():
request.appendBody('%s cannot be found' % basemap[1])
return
raster_layer.append(qgis_layer.id())
qgis_layers.append(qgis_layer)

# Add layer to the registry
QgsMapLayerRegistry.instance().addMapLayers(qgis_layers)
if overwrite:
# Insert all new layers
QgsMapLayerRegistry.instance().addMapLayers(qgis_layers)
else:
# Updating rules
# 1. Get existing layer by name
# 2. Compare source, if it is the same, don't update
# 3. If it is a new name, add it
# 4. If same name but different source, then update

map_registry = QgsMapLayerRegistry.instance()

for new_layer in qgis_layers:
# Get existing layer by name
current_layer = map_registry.mapLayersByName(
new_layer.name())

# If it doesn't exists, add new layer
if not current_layer:
map_registry.addMapLayer(new_layer)
# If it is exists, compare source
else:
current_layer = current_layer[0]

# Same source, don't update
if current_layer.source() == new_layer.source():
if isinstance(new_layer, QgsVectorLayer):
vector_layers.remove(new_layer.id())
vector_layers.append(current_layer.id())
elif isinstance(new_layer, QgsRasterLayer):
raster_layers.remove(new_layer.id())
raster_layers.append(current_layer.id())

# Different source, update
else:
QgsMessageLog.logMessage('Update {0}'.format(
new_layer.name()))
if isinstance(new_layer, QgsVectorLayer):
project.removeEntry(
'WFSLayersPrecision', '/{0}'.format(
current_layer.id()))

map_registry.removeMapLayer(current_layer.id())
map_registry.addMapLayer(new_layer)

if len(vector_layers):
for layer_file in vector_layers:
for layer_source in vector_layers:
project.writeEntry(
'WFSLayersPrecision', '/%s' % layer_file, 8)
'WFSLayersPrecision', '/%s' % layer_source, 8)
project.writeEntry('WFSLayers', '/', vector_layers)

if len(raster_layer):
project.writeEntry('WCSLayers', '/', raster_layer)
if len(raster_layers):
project.writeEntry('WCSLayers', '/', raster_layers)

project.write()
project.clear()
Expand Down
Loading

0 comments on commit ddf4b75

Please sign in to comment.