-
-
Notifications
You must be signed in to change notification settings - Fork 359
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add "is construction finished" quests for highway=construction and building=construction #920
Conversation
Squash changes is something I can do when merging. I just added two icons for you. |
If you use a common class, the maintenance "costs" should be low. |
Thanks for icons, I will add them tomorrow. I think that I will add railway quest as a separate PR, it should be possible by adding new class without reworking general one (can you make icon for railway construction if you want it?). |
So, is this ready for review? |
Regarding railway, are you sure there is a considerate amount of these? |
Yes, as this code goes only icon remains to be fixed. I am not entirely happy about how complex it is, but it is result of handling complex tagging and I see no way to further simplify code (though I probably missed something to make code better, I am curious how it may be improved). |
From looking at taginfo - railway=construction is quite rare and in addition railway routing graph is far less important that road routing graph. That is why I am not sure that railway version is worth adding. |
fabf494
to
b8b6001
Compare
PR was rebased and updated to include new icons (thanks for them!). Code is ready to review, I run out of ideas how to improve it. |
Well… that is obvious even without looking at OSM. Of course, there are more streets than railways, so what is the point? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am impressed! This quest is pretty smart and only possible in this form with this complex overpass query you crafted. All in all, it is in a pretty good shape.
However, there are some open questions. See below.
@@ -27,6 +27,16 @@ public void delete(@NonNull String key) | |||
} | |||
checkDuplicate(key); | |||
changes.put(key, new StringMapEntryDelete(key, valueBefore)); | |||
deleteIfPresent("source:" + key); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This may look like a reasonable thing to do, however, I fear that this will open a can of worms: Source is not the only subtag there is. In fact, there are countless subtags. I.e. if I delete highway=service
, then service=driveway
should also be deleted, if it exists. etc. etc.
Should the app have the entitlement to at this level do this kind of automation, use this kind of intelligence? I think perhaps this should be in the responsibility of each and every individual quest that deletes things.
Another comment: StringMapChanges class (and the Builder) is currently oblivious of any OpenStreetMap data. This line would make it exclusively tailored for OSM which is not reflected in the name of that class.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think perhaps this should be in the responsibility of each and every individual quest that deletes things.
I moved it here as deleteIfPresent("source:" + YET_ANOTHER_KEY) after every single deletion looked ugly, but I failed to consider this. I will move it back. It will make this quest code uglier, but I am no longer sure is it overall improving situation.
EDIT: moved
deleteIfPresent("source:" + key); | ||
} | ||
|
||
public void deleteIfPresent(@NonNull String key) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*IfExists`?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed
import java.text.SimpleDateFormat; | ||
import java.util.Calendar; | ||
|
||
public class DateHandler { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Handler seems to be an odd name for this. I call these classes which only contain utility functions usually *Utils.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed to DateUtil to match QuestUtil
return new YesNoQuestAnswerFragment(); | ||
} | ||
|
||
public static String getQueryPart(String key, String nameOfGeneratedGroup, int rewievIntervalInDays){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this public?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, typo in review
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No idea why it is public, thanks for catching typo. EDIT: handled
" - .known_opening_date_in_future)" + | ||
" - .recently_edited" + | ||
") -> " + nameOfGeneratedGroup + ";"; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is pretty complex, wow! Did you test that this works? Do you think that it could somehow be tested in a unit test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This query could probably be simplified? Perhaps i.e. like this(?):
way[highway=construction](!newer: my-cutoff-date)(if:!is_date(t['opening_date']) || date(t['opening_date'])<date('" + currentDate + "'))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Did you test that this works?", obviously yes, before testing it was full of bugs of various kinds (from invalid queries to not handling unexpected tagging - it even ended with a new tiny feature for overpass turbo :) ).
I was not even thinking about setting up unit test, but it may be a good idea - with query complexity it will be useful as documentation. Main problem is that unit tests rely on OSM data and this kind of data will be especially fragile. But maybe for unit tests we should query historical data that will not change? I just though about it, but it seems that it should solve problem of fragile tests, at cost of even greater complexity.
It is likely that it can be simplified, but I will try to set up unit test before that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool! Well, I was just wondering about the (unit) tests. If to enable testing at all results in (much) more complex code, one should think twice if it is worth it, in my opinion.
You are right of course that this construction data is especially fragile.... perhaps we could use the Airport Berlin as a relatively stable construction site, haha (construction began in 2006, not ready until at least 2020)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, much more complex code is not worth it - but hopefully it can be done without it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
!newer
- unfortunately neither older
or !newer
is an allowed Overpass Turbo syntax. I used other ideas to simplify the query, thanks!
See tyrasd/overpass-turbo#247 for a related feature request
") -> .invalid_construction_type;" + | ||
"(" + | ||
" way[highway=construction][fixme];" + | ||
") -> .with_fixme;" + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are the "(" .... ")" for for the .invalid_construction_type;
and with_fixme query
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, why are fixmes filtered out? (And why then only here, not in the other?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are the "(" .... ")" for for the
I thought that it is a necessary syntax, but I may be wrong. I will try to eliminate it.
Also, why are fixmes filtered out? (And why then only here, not in the other?)
On encountering access tag on the road quest will add fixme and make no further edits. So fixme for roads must be filtered out to avoid repeating quest over and over again and to avoid overwritting existing fixmes. Buildings are not affected by access mess, so it is not necessary to leave fixmes, so it is not necessary to filter them out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see, so that is where that comes from. Well, so on that access
topic, if no feasible automatism can be found, perhaps the user could also be asked about it, since he is already there. So, if the access
tag is not clear from context, the app could ask the surveyor about it after he answered "yes, the construction is finished!".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, if the access tag is not clear from context, the app could ask the surveyor about it after he answered "yes, the construction is finished!
It can be done as a separate issue but given very long list of access tags we run into tag translation issue. I can add it but it would add new translation string for every support access tag (currently list has 19 values).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And it is also trickier than I initially though - what about situation where proper solution is not to accept all access tags or remove all of them, but to remove part of them?
} | ||
} | ||
return false; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very thoughtful!
However, why only include the "most popular" road access tags (why not any, or at least, any documented one)?
And why not remove the access tag?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not any, or at least, any documented one
I though that including all defined at wiki would require either update treadmill (as people invent new ones and add them at https://wiki.openstreetmap.org/wiki/Key:access#Land-based_transportation ) or it would be weird "all common ones and all defined at February 2018".
But I may update list to include all listed at https://wiki.openstreetmap.org/wiki/Key:access#Land-based_transportation - should I do it?
And why not remove the access tag?
https://www.openstreetmap.org/way/83230134 has access tags of way that will be build, https://www.openstreetmap.org/way/540501408/history has something that I am unable to interpret and maybe should be kept maybe deleted https://www.openstreetmap.org/way/88702995 has tags that refer to what will be constructed - for more cases like this see http://overpass-turbo.eu/s/wGm
overall, in this case I prefer to avoid making edit that in 96% will be correct, in 3% will be highly confusing and in 1% will break something and irritate normal mappers. % are obviously based on biased sample and guessing. But even if silent damage, without mistake by mapper, would happen in 0.1% of cases I would still avoid edit like this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I though that including all defined at wiki would require either update treadmill
I mean, why not access=*?
And why not remove the access tag?
Wouldn't then a simpler logic be feasible as well? If access=no|construction_vehicles or similar -> remove access. Anything else, don't change anything?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See for example https://www.openstreetmap.org/way/83230134 where access=no should stay.
I mean, why not access=*?
Just access key? And ignore other access keys? Note that it is quite common to set value also in other keys - foot is the most popular but there are also more complex cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just access key? And ignore other access keys?
Ah, I understand.
return OverpassQLUtil.getGlobalOverpassBBox(bbox) + | ||
getQueryPart("highway", groupName, 14) + | ||
"(" + | ||
"way[construction]" + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't this line superfluous?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, why are "invalid road construction types" filtered out?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had no good idea what to do with cases like highway=construction, construction=service_parking. Throw away construction value? Tag with invalid highway value? And anyway it would further increase complexity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about not excluding it, but treating these kind of invalid values in a way that then the quest tags the fallback highway=road
? I mean, in the wiki, it is pretty clearly defined that the value of the construction tag should be the value of the highway tag, right? So, if it is invalid, then it can be ignored. But what shouldn't be ignored is the fact that the construction site exists at all.
Where does service_parking
come from? Was that just some guy or is this used by several people?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or, another option would be to tag highway=service_parking
in that case. Why should the app fix tagging anomalies automatically that were introduced manually by humans?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so such edit would not introduce a mistake - it may even make it more visible.
And useful data (that construction is completed) is still added.
And it will simplify code.
I think that keeping it makes no sense (it remains from original version of code).
EDIT: Changed
changes.deleteIfPresent(OsmTaggings.SURVEY_MARK_KEY); | ||
} else { | ||
changes.addOrModify(OsmTaggings.SURVEY_MARK_KEY, DateHandler.getCurrentDateString()); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think it would make sense if these lines (53-58) would move to the super class?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These were initially in superclass with other code that I simplified and I moved them here as I though that calling function makes less clear what happens.
I may move them back again (I though about it and was unsure what is better, I am fine with both solutions).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved back to superclass.
groupName + " - .invalid_construction_type)" + | ||
"- .with_fixme" + | ||
") -> " + resultsName + ";" + | ||
resultsName + " out meta geom;"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the getQueryPart
function would accept an additional parameter named something like additionalFilters
, this would simplify this method somewhat
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will try this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code was already simplified by implementing other requests.
I am not convinced that this change would improve it further - it would just move complexity to getQueryPart function that would need to handle additionalFilters.
It would be even worse, as additionalFilters may be empty.
Thanks!
For start - thanks for a review! Simpler changes are committed and pushed - other are added to as TODOs at #920 (comment) (hopefully nothing is missed). Currently my main question is whatever I should add all land access tags listed at https://wiki.openstreetmap.org/wiki/Key:access#Land-based_transportation or keep list as it is current? I plan on handling remaining TODOs and testing code again - hopefully I will be able to do it soon. |
Well, my remark was based on a misunderstanding. I thought you check for |
currently live data is used, as result tests may be broken by OSM edits note new test_unspecified_building_type_is_excluded - this test fails with the curent OSM data, but it is working for specified point of time this increases complexity of code, but it should be preferable to tests failing because OSM data changed I plan on using the same method for unit tests for streetcomplete#920
currently live data is used, as result tests may be broken by OSM edits note new test_unspecified_building_type_is_excluded - this test fails with the curent OSM data, but it is working for specified point of time this increases complexity of code, but it should be preferable to tests failing because OSM data changed I plan on using the same method for unit tests for streetcomplete#920
currently live data is used, as result tests may be broken by OSM edits note new test_unspecified_building_type_is_excluded - this test fails with the curent OSM data, but it is working for specified point of time this increases complexity of code, but it should be preferable to tests failing because OSM data changed I plan on using the same method for unit tests for streetcomplete#920
return overpassServer.getAndHandleQuota(getOverpassQuery(bbox), handler); | ||
} | ||
|
||
/** @return overpass query string to get streets marked as under construction but excluding ones |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
buildings, not streets
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Fixed.
Okay was just a suggestion. My thought was (is) that additional filters would just simplify the query and make it faster because you can limit the results from the start instead of starting out with a bigger result set first and then later having to pluck out parts of it with subqueries+substraction operations.
With additionalfilters I meant simply a string that is appended to the filters in the first statement of the function.
…On 8 March 2018 10:07:23 CET, Mateusz Konieczny ***@***.***> wrote:
matkoniecz commented on this pull request.
> + String groupName = ".roads_under_construction";
+ String resultsName = ".roads_for_review";
+ return OverpassQLUtil.getGlobalOverpassBBox(bbox) +
+ getQueryPart("highway", groupName, 14) +
+ "(" +
+ "way[construction]" +
+ "[construction !~ \"^("+ TextUtils.join("|",
acceptedConstructionValues)+")$\"]" +
+ ") -> .invalid_construction_type;" +
+ "(" +
+ " way[highway=construction][fixme];" +
+ ") -> .with_fixme;" +
+ "((" +
+ groupName + " - .invalid_construction_type)" +
+ "- .with_fixme" +
+ ") -> " + resultsName + ";" +
+ resultsName + " out meta geom;";
This code was already simplified by implementing other requests.
I am not convinced that this change would improve it further - it would
just move complexity to getQueryPart function that would need to handle
additionalFilters.
It would be even worse, as additionalFilters may be empty.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#920 (comment)
--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.
|
The PR has 14 and 180 hardcoded in the getQueryPart calls. Should those perhaps be in ApplicationConstants.java (or one of the new files) instead of magically appearing? |
cdbc43d
to
f5894ce
Compare
Two comments above were mistakenly put by github here, I replied to them in their review threads. |
Ok, so the only open issue is #920 (comment) , then it can be merged. |
22b7ed7
to
bd8cddf
Compare
checks highway=construction checks building=construction fixes streetcomplete#685
a32e670
to
bbef05e
Compare
So at this moment I am again testing after making requested changes. |
DateUtil function getOffsetDateStringFromDate was modyfying passed object now it is fixed
bbef05e
to
f0c6031
Compare
And now, after testing it is ready for a review! Note to self: after merging - push PR for #829 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
Found nothing to complain about anymore. Great job! I trust you tested it, so I will merge it right away. It can go into beta2 that I will release tomorrow or latest on Wednesday. |
@matkoniecz BTW, can you confirm that this overpass query for this quest is still up-to-date? I use it in the wiki. |
The icon looks a bit odd... never seen this type of mark on a construction zone in Germany 🤔 They are white with a white/red reflector here, not yellow black. Maybe a red/white pylon is a bit more internationally recognizeable? 🤔 |
Indeed the colors are more typical for the US, I think. But on the other hand, they are also well-known in Europe (I think), because of websites or so, using this to show "work in progress" or similar things. 😄 |
IMO the icon must not reflect 100% the situation how a sign looks like in every single country because then we end up having 30 icons all meaning the same but having only different colours. Which is way to much effort 😄 |
Well, the black/yellow thing doesn't fit anything familiar and I clicked on it because I was curious what that means. I think a pylon in red/white might be more well known in the most countries. They are even used in the USA, but selecting a barricade which is not used anywhere in Europe is just like asking everybody to insert the maxspeed in USA signs - which we don't do. ;) |
Let's not integrate VLC into this app. 😉 |
@ENT8R I like that, but it really looks like the VLC icon ;) Maybe we can just draw a different style of pylon from a photo? 🤔 |
The result will be the same. You can't get around VLC when drawing a pylon, unless you swap colors somehow or so, but then it looks weird. 😉 |
@rugk what do you think about this one? |
In my opinion current icons are recognizable as construction-related what makes them good enough. I see no necessity of matching them to the most common construction barrier in one specific country. Also, if that would be real problem it would be much better to discuss it in a separate issue - this is a completed, merged pull request - and I prefer to not put it on ignore list because there may be an useful question in future (why I did X?). |
fixes #685
I attempted to test quests quite thoroughly, but I would welcome somebody looking through code and/or trying to break it as it turned out to be more complex that I expected.
To handle:
:)
done:
way[highway=construction](!newer: my-cutoff-date)(if:!is_date(t['opening_date']) || date(t['opening_date'])<date('" + currentDate + "'))
in superclass will work as well as the current monster?unit tests TODOs:
new building=construction with opening_date in future is not triggering the quest= dropped, de facto tested by highway testing and this tagging is rare - I failed to find even single correctly tagged building with opening_date that was note edited in last half of year...