osrm-routed consistent Segementation fault on specific requests

[thomaswilkins]

When running osrm-routed on Ubuntu 12.04 LTS on an Amazon EC2 instance some requests are consistently causing segmentation faults. For example:

[info Server/RequestHandler.h:65] 25-06-2013 09:30:41 92.19.231.90 - Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0 /viaroute?loc=51.53649,0.06923&loc=51.52253,0.06552
Segmentation fault (core dumped)

This is using the europe-latest.osm.pbf form GeoFabrik and the latest source code from the Git repository as in the build instructions.

git clone https://github.com/DennisOSRM/Project-OSRM.git

I compiled with the --no-march flag.

[DennisOSRM]

A couple of things were fixed in the mean-time. Is this issue still reproducable in the develop branch on your end? Also, please note that builds are done in cmake now instead of scons.

[drieks]

Hi,

I can confirm that it works for me now with latest develop branch.

Dennis

[DennisOSRM]

Ok, cool. Thanks for the feedback.

[sam-robinson]

I am getting this same problem on an Amazon EC2 Ubuntu 13 instance (m2.4xlarge, over 64GB memory). Downloaded source today and north_america_latest from Geofabrik. I've had this problem for over a week trying both north america and the planet, and have not been able to resolve it.

Here's a simple run - it works for the first query (which is about 5 blocks in Denver, CO) while it segfaults on the second (which goes from north to south in Denver):

[server] running and waiting for requests
[i /osrm/Project-OSRM/Server/RequestHandler.h:57] 03-08-2013 21:51:38 127.0.0.1 - curl/7.29.0 /viaroute?loc=39.740140,-104.890680&loc=39.740150,-104.903080
[i /osrm/Project-OSRM/Server/RequestHandler.h:57] 03-08-2013 21:51:45 127.0.0.1 - curl/7.29.0 /viaroute?loc=39.740140,-104.890680&loc=39.812888,-105.033588
Segmentation fault (core dumped)

I followed the basic build instructions in the wiki. Any ideas?

[drieks]

Hi,
did you use "mkdir build ; cd build ; cmake .." or "cmake ." for compiling osrm?
First I tried "cmake .", later the first version. If you use "cmake ." can you try this version? "mkdir build ; cd build ; cmake .."

[emiltin]

or if you have ruby installed, you can just use 'rake' which will make the build dir if needed, and build inside it (like suggested above)

[emiltin]

should this be reopened?

[drieks]

Hi,

it works fine, at least for me. I thought rake was replaced by cmake?

[sam-robinson]

I used

mkdir -p build; cd build; cmake ..; make

Extract and prepare ran without issue.

When you say it works for you, is this on an Amazon EC2? If so what are the machine specs?

Thanks for the help!

[emiltin]

no, scons was replaced by cmake.
"rake" is just a shortcut for "mkdir -p build; cd build; cmake ..; make"

[DennisOSRM]

@emiltin is right, scons was replaced. rebuilding the source is not necessary in general, but AWS/EC2 has a number of oddities.

E.g. you have no control on what physical hardware your virtual machine runs. When compiling, the compiler applies certain optimizations that are CPU specific, i.e. it uses the features that are advertised by the virtual CPU. It appears that this is very dependent on what kind of real hardware you are running. When you downsize/reboot/start your VM, it may very well be started on different hardware. So the previous compiler optimizations become invalid and thus stuff may not work anymore.

[drieks]

@roblysam: are you on develop branch? This bug is not fixed on master branch yet...

[sam-robinson]

@drieks I assume I'm on the dev branch, I used the following to get the code:

git clone https://github.com/DennisOSRM/Project-OSRM.git

[drieks]

you must use this clone command:
git clone -b develop https://github.com/DennisOSRM/Project-OSRM.git

you can switch to develop branch using this command:
git checkout -b develop -t origin/develop

[drieks]

(you are on master, check "git status")

[DennisOSRM]

or git branch

[sam-robinson]

thanks I will try that

[thomaswilkins]

Does the new build system offer the option to disable architecture specific optimisations as scons did? This may help with some of the issues on EC2.

Additionally is there any timetable for including this bugfix in the master branch?

[benjamin-rty]

I have the same problem as Roblysam 5 months ago.
Even on the same requests:

[server] running and waiting for requests
[i /osrm/Project-OSRM/Server/RequestHandler.h:57] 03-08-2013 21:51:38 127.0.0.1 - curl/7.29.0 /viaroute?loc=39.740140,-104.890680&loc=39.740150,-104.903080
[i /osrm/Project-OSRM/Server/RequestHandler.h:57] 03-08-2013 21:51:45 127.0.0.1 - curl/7.29.0 /viaroute?loc=39.740140,-104.890680&loc=39.812888,-105.033588

I use OSRM 3.5 and have 64GB ram, so I do not think thats the problem.

What's the solution ?