Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem installing Nominatim with France pbf #580

Open
blaurent49 opened this issue Sep 12, 2024 · 9 comments
Open

Problem installing Nominatim with France pbf #580

blaurent49 opened this issue Sep 12, 2024 · 9 comments
Labels

Comments

@blaurent49
Copy link

Describe the bug
Hello,
I tried to install Nominatim on a Debian VM (500 GB of hard disk available) via Docker with the France-latest.osm.pbf file previously downloaded from https://download.geofabrik.de/europe/.

Here is the command I used:
sudo docker run -it
-e PBF_PATH=/nominatim/data/france-latest.osm.pbf
-e REPLICATION_URL=https://download.geofabrik.de/europe/france-updates/
-e POSTGRES_SHARED_BUFFERS=2GB
-e POSTGRES_WORK_MEM=64MB
-e POSTGRES_MAINTENANCE_WORK_MEM=1GB
-p 8001:8080
-v $PWD:/nominatim/data
--name nominatim
mediagis/nominatim:4.4

The process started perfectly. It has been running for hours (at least 10h !) until encoutering the following error:
psycopg2.errors.Diskfull: could not resize shared memory segment "/PostgreSQL.4111479138" to 50438144 bytes: No space left on device.

I don't understand the space problem because the VM has 500 GB of hard disk available.

I tried to check the posgresql.conf file but the shared_buffers, work_mem and maintenance_work_mem are in comments or at their default value. How can I be sure that the environnement variables are taken into account?

Am I doing something wrong?

Thank you in advance for your help.

Screenshots & Logs
Capture d’écran 2024-09-10 225441

Desktop / Server (please complete the following information):

  • OS & Version: Debian 12
  • Docker Version: 27.2.0
  • Nominatim Version: 4.4
@blaurent49 blaurent49 added the bug label Sep 12, 2024
@achedeuzot
Copy link

Hi @blaurent49
Have you checked if your docker/postgres installation has access to the full size of your disk ?
I had this issue in the past because docker couldn't use all the space available: and the error you're getting is quite explicit about it.
Can you run df -h to check how much free space is available ?

For France latest, 175GB should be enough 😉 Depending on your RAM / CPU, it can take many hours.
For a 8 CPU + 16GB of RAM server, it took around 18 hours.

@blaurent49
Copy link
Author

Thank you for your answer.

Here is the result of the dh -h command:
image

As you can see, there should be enough space, shouldn't it?

@achedeuzot
Copy link

@blaurent49
I figure that the VM or mapped space can't use the whole disk you have configured. I'd look into your setup to fix this issue.
Best of luck 🙂

@blaurent49
Copy link
Author

What can I look, more precisely?

@achedeuzot
Copy link

@blaurent49
I think the issue is not on nominatim but in your storage setup. You'll have to check why your nominatim container can't use more storage (software limit somewhere?). I would look into that first 🤔

Also, have you run the 'df -h' command from inside the nominatim container?

@blaurent49
Copy link
Author

I ran a bash in my container with docker exec -it nominatim /bin/bash and ran the df -f command from inside it.

Here is what I got:
image

It seems all right at this stage, doesn't it?

@blaurent49
Copy link
Author

blaurent49 commented Sep 20, 2024

Hello,
I eventually solved my problem. Here is the command I used:

docker run -d \
  -e PBF_PATH=/nominatim/data/france-latest.osm.pbf \
  -e REPLICATION_URL=https://download.geofabrik.de/europe/france-updates/ \
  -e POSTGRES_SHARED_BUFFERS=2GB \
  -e POSTGRES_WORK_MEM=64MB \
  -e POSTGRES_MAINTENANCE_WORK_MEM=1GB \
  -p 8080:8080 \
  -v $PWD:/nominatim/data \
  --name nominatim \
  --shm-size=2gb \
  mediagis/nominatim:4.4

That wasn't a Nominatim related problem. It came from Docker: the containers by default have a shared memory (/dev/shm) size of 64MB. The important part in my command is: --shm-size=2gb

After that, I could install Nominatim. I didn't time precisely but it may have taken 24h. At least, it's a victory!

@GildasLepennetier
Copy link

Thanks @blaurent49, I also had an error at this step with a error message that was not helping.
Using your extra options, I can reach the end of the import of the data into the database 👍.

@achedeuzot
Copy link

Interesting, it's in the default options recommend in the docker documentation but I didn't know it would return a storage issue. Good find!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants