Can't install shape files while building docker

[patricerosay]

Hi.
I used to be able to install the docker, but now, I'm running into the following issue:
I'm downloading files from git
I execute the command "make test"

=>There's an error while downloading the file:

http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip

I've tried to upgrade manually the Dockerfile to the newest version openstreetmap-carto 5.3.1 without success so far.

Please advise.
P.

[Istador]

The external URL isn't available anymore and therefore it fails.

The newest carto version 5.3.1 changed the URL to use another source:

• https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip

But they also renamed the script name that needs to be called with version 5.3.0 from scripts/get-shapefiles.py to scripts/get-external-data.py. So, that needs to be changed in the Dockerfile too:

openstreetmap-tile-server/Dockerfile

Line 130 in 75e7714

&& scripts/get-shapefiles.py \

[patricerosay]

Hi, Thank you for your answer. I tried that already. Installed pip to get the python3’s package update to make the scripts/get-shapefiles.py run. But new errors are occurring. Traceback (most recent call last): File "./scripts/get-external-data.py", line 294, in <module> main() File "./scripts/get-external-data.py", line 192, in main psycopg2.connect(database=database, File "/usr/lib/python3/dist-packages/psycopg2/__init__.py", line 127, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"? The Postgres needs to be running and configured. # ./scripts/get-external-data.py Traceback (most recent call last): File "./scripts/get-external-data.py", line 294, in <module> main() File "./scripts/get-external-data.py", line 192, in main psycopg2.connect(database=database, File "/usr/lib/python3/dist-packages/psycopg2/__init__.py", line 127, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: FATAL: role "root" does not exist Are you planning an update of your package ? P.

…

Le 28 avr. 2021 à 18:44, Robin C. Ladiges ***@***.***> a écrit : The external URL isn't available anymore and therefore it fails. The newest carto version 5.3.1 changed <gravitystorm/openstreetmap-carto@396778b#diff-cea0c5554020ead8c74ea96861663b7c1a8cf4058ef9e9b0f2d58b8a38e1947a> the URL to use another source: https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip <https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip> But they also changed the script name <gravitystorm/openstreetmap-carto@6782400#diff-40def9a8c3707b1957f5cec30f808df5518f7ae5528d681822a3abd9fb582d6d> that needs to be called with version 5.3.0 from scripts/get-shapefiles.py to scripts/get-external-data.py. So, that needs to be changed in the Dockerfile too: https://github.com/Overv/openstreetmap-tile-server/blob/75e7714f200cf452bfbe2f249453125632773e48/Dockerfile#L130 <https://github.com/Overv/openstreetmap-tile-server/blob/75e7714f200cf452bfbe2f249453125632773e48/Dockerfile#L130> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#181 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA5BOOBA35CLADVMJQKQTPTTLA3ONANCNFSM43XHWP6Q>.

[Istador]

Ah, ok, it needs the database running.
Then I'd suggest to not call it inside of the Dockerfile, but in the run.sh somewhere during the import command (probably after the first init block?):

openstreetmap-tile-server/run.sh

Lines 36 to 83 in 75e7714

	sudo -u postgres createuser renderer
	sudo -u postgres createdb -E UTF8 -O renderer gis
	sudo -u postgres psql -d gis -c "CREATE EXTENSION postgis;"
	sudo -u postgres psql -d gis -c "CREATE EXTENSION hstore;"
	sudo -u postgres psql -d gis -c "ALTER TABLE geometry_columns OWNER TO renderer;"
	sudo -u postgres psql -d gis -c "ALTER TABLE spatial_ref_sys OWNER TO renderer;"
	setPostgresPassword
	# Download Luxembourg as sample if no data is provided
	if [ ! -f /data.osm.pbf ] && [ -z "$DOWNLOAD_PBF" ]; then
	echo "WARNING: No import file at /data.osm.pbf, so importing Luxembourg as example..."
	DOWNLOAD_PBF="https://download.geofabrik.de/europe/luxembourg-latest.osm.pbf"
	DOWNLOAD_POLY="https://download.geofabrik.de/europe/luxembourg.poly"
	fi
	if [ -n "$DOWNLOAD_PBF" ]; then
	echo "INFO: Download PBF file: $DOWNLOAD_PBF"
	wget "$WGET_ARGS" "$DOWNLOAD_PBF" -O /data.osm.pbf
	if [ -n "$DOWNLOAD_POLY" ]; then
	echo "INFO: Download PBF-POLY file: $DOWNLOAD_POLY"
	wget "$WGET_ARGS" "$DOWNLOAD_POLY" -O /data.poly
	fi
	fi
	if [ "$UPDATES" = "enabled" ]; then
	# determine and set osmosis_replication_timestamp (for consecutive updates)
	osmium fileinfo /data.osm.pbf > /var/lib/mod_tile/data.osm.pbf.info
	osmium fileinfo /data.osm.pbf | grep 'osmosis_replication_timestamp=' | cut -b35-44 > /var/lib/mod_tile/replication_timestamp.txt
	REPLICATION_TIMESTAMP=$(cat /var/lib/mod_tile/replication_timestamp.txt)
	# initial setup of osmosis workspace (for consecutive updates)
	sudo -u renderer openstreetmap-tiles-update-expire $REPLICATION_TIMESTAMP
	fi
	# copy polygon file if available
	if [ -f /data.poly ]; then
	sudo -u renderer cp /data.poly /var/lib/mod_tile/data.poly
	fi
	# Import data
	sudo -u renderer osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script /home/renderer/src/openstreetmap-carto/openstreetmap-carto.lua --number-processes ${THREADS:-4} -S /home/renderer/src/openstreetmap-carto/openstreetmap-carto.style /data.osm.pbf ${OSM2PGSQL_EXTRA_ARGS}
	# Create indexes
	sudo -u postgres psql -d gis -f indexes.sql
	# Register that data has changed for mod_tile caching purposes
	touch /var/lib/mod_tile/planet-import-complete

When you get it to work again with the new version, you can send in a pull request with your changes. @Overv will probably be happy to merge the fix.

---

There is already pull request #176 that tries to update carto to 5.3.1. But if the get-external-data.py needs access to the running database, then the PR should not work correctly.