Working sample configs for oscar-create #37
Comments
|
Hi, Depending on what you're planing to do I do want to stress the fact that oscar-create needs a lot of memory if you intend to process larger files. |
|
I appreciate it! No rush at all. I will check out the docker as well. If it
has a usable config perhaps it would work for me. I don't think my project
would work with docker though, and at any rate I'd like to be able to build
everything up from scratch.
My project will probably be memory sensitive, that's useful info. However I
expect the use case to be more on the scale of a US state.
…On Sat, Sep 11, 2021 at 8:49 AM dbahrdt ***@***.***> wrote:
Hi,
I'll take a look into the crash as soon as possible (however I'm on
vacation for the next 14 days and I don't know how much time I'll find to
address this).
Maybe you can try to take a look at
https://github.com/dbahrdt/oscar-docker?
This will give you a whole installation of oscar and oscar-web with which
you can build the necessary files.
The docker container also has support for routing.
I'll try to come up with a small sample configuration so that you have
something to work with.
Depending on what you're planing to do I do want to stress the fact that
oscar-create needs a lot of memory if you intend to process larger files.
In order to process the planet data set you'll need about 256 GiB of RAM.
However you may find pre-processed files at http://data.oscar-web.de. The
latest version is from February since I started porting all size related
stuff to uint64_t due to OpenStreetMap containing too many items for oscar
to process.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#37 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAKH6DUAR5YMHONFKMIATTUBNF6LANCNFSM5DYLVUIA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
I'm about to look at this project again. l figured I should let you know a little more about what I'm working on in case you can give me other suggestions. For the time being, I will see if the Docker project gives me something to work with. In the end I'd like to be able to build everything from source, though. The project I'm working on is to get a fully self-hosted web-based map, letting the user choose the region. (Either pre-defined regions or arbitrary regions, whatever is easier). I'm trying to keep the app as simple and self-contained as possible. With the environment I'm targeting for the app's server, running a database like Postgres (that usually requires multiple user accounts) is complicated. Amazingly I figured out a solution for the map part (Protomaps), but I still need a search function. Oscar looks like a single-user solution to the search issue, which is just what I need. So what I think I need to do now is to prepare (before the user installs the app) Oscar files from raw OSM data of every desired region that the user might select from. I could host these files on a server, and the user's app would download them as the user select the given regions. The Oscar file generation step can be run in a normal Linux environment, with fewer resource constraints than the app server. |
|
FYI I found an alternative for my needs, so don't spend any extra energy for my sake at this point. Leaving this open though since it could be worth looking at in general. |
|
Sorry for my unkind long delay. I just don't have the time to take care of external issues of this project anymore. It might be helpful for other persons that stumble upon OSCAR, if you could comment what alternative you found. |
|
I'm using Gazetteer to extract the data. The output is a list of json objects. I'm extracting what I need from it (which is not much, yet) into sqlite fts5. It's a hack, and probably won't compete with a proper OSM search engine, but it's good enough for now. That said, Gazetteer crashes on some data sets too. I'm hoping I can get some help from them. I'm not sure about their availability either. Anyway, thanks for chiming back in. BTW if you're curious what I'm working on: https://github.com/orblivion/sandstorm-share-a-map |
I'm trying to get oscar-create to work for a sort of proof-of-concept project. I'm looking for any working configuration, even if it's very basic, and then experiment from there if I need any adjustments. I tried working with
sampleConfig.json, but I had issues with it, detailed below.So, what I'm wondering is, do you have a working set of config files handy, that you would not mind sharing? Or if you know what I did wrong with my config, that would work for me as well. Or, maybe the segfault is just an unrelated bug.
Thank you!
As for what I tried:
I started with
sampleConfig.json. It lists options, but I didn't see documentation for it, so I sort of had to guess as to how to get the simplest version to work. I have it successfully parsing now, but from there it ends in a segfault.My config.json
empty.json(used in myconfig.json):empty-obj.json(used in myconfig.json):My input file (~11MB):
https://download.geofabrik.de/north-america/canada/prince-edward-island-latest.osm.pbf
The output:
The text was updated successfully, but these errors were encountered: