Very few quests are fetched

[RubenKelevra]

When I open the app the first time, there are like 10 quests within 500 meters of my location fetched and displayed.

Only after 2-3 manual fetches, a lot of additional quests pop up.

House type quests for example doesn't pop up at all without manual fetch.

Is this on purpose?

To me it looks like just a certain subset of quests is fetched on each refresh - which is quite odd.

A new user might be confused, since he has to walk 500 meters to the first pin, while there are a lot of quests around him.

How to Reproduce

• Clear the storage of the app and open it.
• Wait until the refresh is finished.

What I expect to happen?
All possible quests are loaded for the area.

What happens instead?

Just a small number of quests-types are loaded.

Versions affected
Android 9
StreetComplete 19.2

[westnordost]

Yes, it is not a bug but deliberate behavior.

On mobile data, the app downloads 4 quest types within a radius of 600m around you first. Only if there are less than 8 quests per km² in a radius of 300m around you, it will download another 4 quest types, and so on.

If you are on wifi, it will download 8 quest types within a radius of 1200m around you first. Only if there are less than 12 quests per km² in a radius of either 300m or 600m around you, it will download another 8 quest types, and so on.

These values are not set in stone, we can discuss this. I just want to tell you how the app decided when to download more and when to leave it be.

[westnordost]

if there are less than 8 quests per km² in a radius of 300m around you

Hmm, so that's an area of PI*0.3*0.3 = 0.28 km², so the requirement is to have at least 2.26 quests within 300m of the current location. In average, this means that the download stops after at least 9.04 quests have been found in a radius of 600m.

[RubenKelevra]

I think we should focus more about very near pins.

Currently quests get downloaded which are like 40 walk minutes away (2.54 km of walking distance in a nearly straight line).

I'm fine if the download area is like 100m radius - which should be increased if the GPS-Speed is above say 10 kph - that's mapping from a vehicle.

I understand that limiting the amount of tasks can be useful as a feature, since many people have limited data usage. How about not restricting the tasks unless the 'data saver' option on Android is activated? 🤔

And maybe add a manual override...

Limit datausage:

• Automatic
• On Mobile connection
• Always

[westnordost]

A small download radius does not make much sense because the time it takes for the server (overpass) to answer is the same for very very small areas, very small areas and small areas. But what we can change is the "active radius", the radius which decides over whether new quests should automatically be downloaded.
I didn't know about the data saver option. Is it new?

[smichel17]

Is it new?

Added in 7.0, I think. (aka Nougat, aka api 24, for anyone like me who can't keep them straight and needs to keep the page with the table bookmarked)

[RubenKelevra]

A small download radius does not make much sense because the time it takes for the server (overpass) to answer is the same for very very small areas, very small areas and small areas. But what we can change is the "active radius", the radius which decides over whether new quests should automatically be downloaded.

Okay, this makes more sense then.

Within a 600m x 600m box around my GPS position there are like 3 pins, maybe 4.

I think if there are no pins within a 100m x 100m box around the GPS position, StreetComplete should start finding more stuff (even with the discussed data saving on).

The nearest pin downloaded after a fresh installation is 0.38 km away (walking time 7 minutes) - very discouraging for new users. If StreetComplete would continue downloading, it would find dozens of quests in this area.

I didn't know about the data saver option. Is it new?

I guess. My oldest Android is 8.1 on the Tablet and it has this option.

You can override the data saving in the 'App info', under 'App data usage'.

With Android 9 it's possible to set a WiFi to be metered/unmetered too.

Anyway - this is only interesting for the "automatic" option, which should be standard if the API supports this info.

I just like to override the data saving options completely, I don't have a metered mobile connection and just want all quests in the area I'm in as an option - if that's possible. :)

[HolgerJeromin]

With Android 9 it's possible to set a WiFi to be metered/unmetered too.

This is respected since #1699

[westnordost]

I adjusted the values a bit

[RubenKelevra]

I adjusted the values a bit

Can we get an option in the settings to skip all those limitations for mobile usage? Unlimited data on mobiles are a thing and I really don't care how much data is used. I just want to see all available quests to avoid having to walk a street twice.

[westnordost]

Well, not an option, but you know the "scan for quests here button". With this, you can effectively download all the quests in an area manually. The (mobile) data usage is not the only issue but also the app should not use too much capacity of the Overpass API server

…

On May 20, 2020 3:44:29 AM GMT+02:00, ***@***.***" ***@***.***> wrote: > I adjusted the values a bit Can we get an option in the settings to skip all those limitations for mobile usage? Unlimited data on mobiles are a thing and I really don't care how much data is used. I just want to see all available quests to avoid having to walk a street twice.

[RubenKelevra]

It's not that easy. To fetch all quests for an area you have to wait for the initial download to complete, and then use the manual download trigger 7 times to get all quests for an area.

Depending on the respond time that's like 1 and a half minute of manual interaction just to get all quests for an area. Time I could have used to map instead.

I understand that we don't want to overload the API by having this on by default, but why not let users choose to download everything in one go?

As far as I understand all quests would be downloaded anyway if the user walks around in an area for a longer period of time.

The issue with the current approach for me is: I want to clear all quests in an area I traveled to. When I use the automatic I would have to walk to the same spots multiple times, since there would constantly new quests pop up in areas which I already cleared before.

That's not following the Rule of Least Surprise but instead also pretty frustrating for the user.

[smichel17]

The (mobile) data usage is not the only issue but also the app should not use too much capacity of the Overpass API server

Could that be addressed by writing one big query to sweep up all the quests at once, instead of individually per quest, and then sorted by quest type on-device? Or would that be too much on-device processing?

[westnordost]

Could that be addressed by writing one big query to sweep up all the quests at once, instead of individually per quest, and then sorted by quest type on-device? Or would that be too much on-device processing?

The app could do a normal OSM API query for an area to sweep up all quest types that are possible to sweep up with that (basically those that do not return null on isApplicableTo) and only use Overpass for the more complex ones. PR welcome, as this may speed up download time as well.

[matkoniecz]

Could that be addressed by writing one big query to sweep up all the quests at once, instead of individually per quest, and then sorted by quest type on-device? Or would that be too much on-device processing?

I see no sane way to get multiple quests to use one query together - which ones would qualify?

Combining multiple queries into one super query with subqueries AFAIK would not reduce processing for Overpass Server at all. But it would increase waiting tome and how often query is rejected or fails.

Fetching all OSM data and filtering on phone would require absurd amount of data use and such processing is unfeasible on a phone anyway. And it is dubious whatever it would reduce Overpass Server resource use.

[westnordost]

I see no sane way to get multiple quests to use one query together - which ones would qualify?

https://wiki.openstreetmap.org/wiki/API_v0.6#Retrieving_map_data_by_bounding_box:_GET_.2Fapi.2F0.6.2Fmap

Fetching all OSM data and filtering on phone would require absurd amount of data use and such processing is unfeasible on a phone anyway.

Are you sure it's that much? I'd think it ranges somewhere between 1-5MB for the areas we are talking about.

[Atrate]

I'd say processing all the OSM data might actually be a good idea since it's really not that much in terms of size for not huge areas (unless you have to DL a coastline relation)

[RubenKelevra]

@matkoniecz

Fetching all OSM data and filtering on phone would require absurd amount of data use and such processing is unfeasible on a phone anyway. And it is dubious whatever it would reduce Overpass Server resource use.

I tried to back this up with some numbers:

OSM download traffic for a typical area SC will download (in the city): 600 KByte.

Manually download all quests for an area (plus the default zoom level map after installation): 1,06 MB.

Filesize of the OSM data after decompression is 6.4 MB.

It doesn't sound that absurd to download the full OSM data for an area and then filter on the device.

I like the distributed approach, it scales much better.

[matkoniecz]

Thanks for checking! I am quite surprised by this result.

[HolgerJeromin]

We could download all data from osm api or overpass.
This will reduce load on the overpass api due to missing processing.

[RubenKelevra]

Is there a way to check if there has been changes to an area since a given timestamp?

This way we could avoid redownloading areas which haven't been updated.

If there has been updates, we could split the area in 4 once or twice to reduce the update size, if there's just a single update.

Would be cool to create a project out of this and nail down what has to be done to do local processing.

@westnordost do you think it's viable to build an osm to svg converter in the future, to avoid having to store the same data twice?

Vespucci does live rendering of OSM data pretty performant on devices as well.

@westnordost please reopen, since it's pretty pointless to discuss in a closed ticket :)

[westnordost]

Is there a way to check if there has been changes to an area since a given timestamp?

No. But that something in an area has been updated is very very likely, much more likely than when sparse-downloading via Overpass.

@westnordost do you think it's viable to build an osm to svg converter in the future, to avoid having to store the same data twice?

SVG? Ah I see what you mean. It would need to be an OSM to MBTiles converter. MBTiles in the same format as map-data.de/nextzen. @Akasch might know iif there is something usable for Java.

@westnordost please reopen, since it's pretty pointless to discuss in a closed ticket :)

I'd rather have you open a new ticket because this is about something different now than how it started.

[RubenKelevra]

@westnordost well, the initial issue still persists in the v20 stable:

First start at a random location in London. And the nearest quest automatically downloaded is like 2 blocks away.

Can we reduce the area where SC counts the download quests to like say 100x100 meter? So 50m radius, and if there are not something like 10 quests, we start downloading more?