Consensus based decision making #3951
Comments
BackgroundIf I understand the history correctly, back when this style started, @gravitystorm created the original design based on the old XML stylesheets used on openstreetmap.org previously, with the help of a few others. Several other people were made maintainers of the style, and there was an rule (perhaps unwritten?) that all changes should be approved by one of the maintainers prior to being merged. If there were objections, they needed to be addressed. In April 2017 there a change which allowed maintainers to merge their own PRs even if there was a negative review requesting changes:
However, a review was still required before merging. Then in July 2017 there was a suggestion that maintainers should be allowed to merge their own changes even without a review (Perhaps after a 1 month wait #2436 (comment)) There was not consensus in favor of this change, see #2436 (comment) and following comments, however, over the next year (late 2017 to late 2018) many PRs were merged without a full review. Then in December 2018, @gravitystorm responded, indicating that he thought this should be reconsidered:
Later it was clarified that:
Also see #2436 (comment) I believe this history gives a clear outline to what we mean by consensus-based decision making. |
Additional commentsIn addition, there have been some other thoughts about how this should work: Reviews by everyone
(This last point needs to be discussed further - perhaps we should try to give reviews by everyone an equal hearing?) I certainly support encouraging more reviews by all contributors, including first-time contributors who might be interested in a particular topic. Using Github review function
I agree with this idea. If a reviewer choses not to use the official "review" function and "request changes", it can be hard to remember a few weeks later. This also makes it easier to tell when a critical review has been resolved. If the review is submitted as a comment only, it should be very clearly stated that "I request changes" or "I still do not approve of this change, because..." - especially if there have been further comments and discussion that may have appeared to resolve the objection. When objections are serious enough to block a change
This is a point we need to discuss, which has come up recently in #3923 (where @matkoniecz had some reservations but chose to let the PR be merged), #3855 (@imagico offered a better solution, but has not specifically requested not merging the change), #3930, and #3851. There was a RFC on 'rough consensus' which was discussed in the prior issue. See this summary: #2436 (comment)
This means that there needs to be a way to decide when an objection is significant, and when it is a matter of taste, or subjective preference, which should not prevent the change. |
How should we deal with changes which were merged without consensus?I recently accidentally merged #3930 over the objections of another maintainer. I thought the objections had already been addressed, and because there had not been any additional comments after the PR was approved by a different maintainer. There were also a large number of previous PRs in late 2017 and throughout 2018 which were merged in spite of specific objections from maintainers or other, and several that were merged without a final review. Should we consider "the past is in the past" and move on, considering request to revert such changes the same as any new PR, or should there be a specific way to handle these cases? |
|
I think that PRs that have clear "it should not be merged because XYZ" from maintainer should not be merged. It would be preferable to do it using explicit GitHub review function so it would not be missed. In case of mistaken merging because of misunderstandings revert seems a good idea. In an ideal world I would support at least one mandatory review. But I still plan to make some pr reviews and then end not doing this, so I am not going to request making reviews mandatory. |
|
For productive cooperation i think it is also significant that objections are always combined with a clear vision on how to move forward. Most PRs are made with the aim to address some problem. An objection to such a PR should preferably offer a clear positive perspective how to deal with the problem meant to be addressed by the PR. As i have said before i think for productive cooperation in a project like this it is important that consensus building is based on arguments and reasoning with the aim to come to a common position and strategy on design. If consensus building is interpreted as being about negotiating compromises between different conflicting directions this can't work in my eyes. To put it bluntly: The compromise between showing something on the map and not showing it will always be bad map design. |
|
And yes, i think normally PRs merged without consensus should be reverted - but specifically in this case it does not hurt probably to first assess if there is indeed a sustained lack of consensus. |
|
I would also like to remind everyone that consensus based decision making can only work if everyone is actively working towards a consensus and puts the goal of finding a common direction and strategy in design above personal subjective interests and likes and dislikes. @gravitystorm already formulated this in what has been cited above:
I have seen in more recent discussions a distinct lack of willingness to work towards a common understanding of the direction of this style. This seems to be the case in #3930 but also for example in #3855 and especially also in #3750 - and of course also in #3635. How written guidelines can help in establishing and documenting a common understanding of our strategy has been pointed out in what i cited above. But if even the most fundamental goals as we have them documented are in discussion treated mostly as obstacles to route around in pursuit of subjective personal interests that is not going to work in the long term. The discussion and in parts very fundamental disagreement on strategy that shows in the PRs linked to above has not lead us to work on refining our guiding principles and resolving these disagreements. |
|
I would still like to hear from all the maintainers and other active contributors who have thoughts about this issue. We should agree to a process for consensus-based decision making and document this in CONTRIBUTING.md or another appropriate location. How should we resolve cases when two maintainers disagree? Should a well-argued review by any contributor should be given equal weight? How do we define when consensus has or has not been achieved? |
|
I have created a new consensus needed label and started assigning it to those issues/PRs where progress is blocked by the lack of consensus among maintainers. |
|
@kocio-pl what are your thoughts? |
|
Some general thoughts: Recently my view on this topic has risen from low to medium. I think any direct procedure crafting is too fast at this moment. From my work experience, procedures are deployed for given organization and cannot be easily transferred from any other and it starts with identifying organizational structure and culture. I don't like to make long, vague comments about state of the project - I think #2291 (comment) is a good starting point for this and it looks more promising now. |
|
It sounds like you are not happy with making decisions based on consensus, as described above (#3951 (comment)): "Build consensus and agreement through discussion and compromise, using a combination of the cartography guidelines and common sense." "Consensus doesn't mean unanimity, but there shouldn't be anything merged here against firm opposition from a maintainer. On the other hand, a maintainer shouldn't block a change without offering a very good reason, or another way forward." What parts of this do you think are causing problems? Is there an alternate way of making decisions which would be feasible? |
|
I really think this is not something you should start with exact solutions (it would be a part of the problem for me). If I was to describe the source of problems in a nutshell, it would be narrow team with different priorities and visions, which is of course hard to solve in a direct way. |
|
Does that mean that the method of consensus-based decision making described above is a good idea, but we are having problems because of the current team members, specifically our different priorities and goals? Question: are we actually following the description of consensus-based decision making above? Examples: " a maintainer shouldn't block a change without offering a very good reason, or another way forward" "If there's a review that highlights some problems with the PR, those problems must be resolved" "In a consensus based decision making system it is essential to not exercise your right to disagree when it is not fundamental for you. ... expect objections to a change to be taken into account and not be overruled. But this in return means ... let others who have a different opinion make decisions I disagree with." #3951 (comment) My impression is that changes are sometimes being blocked due to individual preferences rather because of specific objections which are based on the cartographic guidelines or specific problems with the proposed solution, and alternative solutions are not always being offered or accepted. |
|
What I see above is exactly an example of crafting fast solution without actually deploying it to the real project, which in my opinion is going to make the problem worse. If the root cause is "narrow team with different priorities and visions", then you end up with different "leaf" problems which will pop up in different places again. So at this moment I will go to the queue of things that are either easier to fix or more basic than this and I plan to come back here later. |
|
@kocio-pl - just as communication feedback to you: I have quite extreme difficulty understanding your statements here. I concur with @jeisenbe that it seems you are not happy with the idea of consensus based decision making but you don't seem to be comfortable with even discussing the topic. Consensus based decision making requires us (the maintainers) to face and resolve our differences and come to a common understanding of the goals and direction of the project. If we can't do that we will not be able to fulfill our function in the long term (and i mean both we, the maintainers, not fulfilling our function within the project as well as OSM-Carto not fulfilling its function within the OSM community). It is completely fine in my eyes that some of the maintainers are at the moment not able to invest significant amount of time to the project and therefore can't contribute much to consensus building and finding a common direction as long as this does not hamper building consensus among the rest of us. But those of us who actively contribute to decision making at the moment have an obligation IMO to actively contribute to consensus building. If it is still unclear what this entails - @jeisenbe already explained this quite in detail i think - please say so. |
|
Wanted to comment what is written in #3977 (comment) - doing it here since it is more fitting. Please keep in mind this is primarily a cartographic design project, not a software development project. All of the issues where we currently lack consensus are due to disagreements on cartographic matters, not on technical matters. In software development you can make unit testing, define interfaces between different components and this way can compartmentalize and deal with problems individually. In cartographic design you always have to look at the big picture since map users look at and perceive the map as a whole thing. Also keep in mind that we have tried the approach of allowing changes without consensus and without substantial review for more than a year. The results of this were primarily a large number of simple feature additions based on specific (urban, European) special interests but a strong neglect of overall cartographic design development, maintenance and a failure to address any of the larger problems (both cartographically and technically). I have pointed out a long time ago in #2270 a clear shared vision and cartographic paradigm is essential to motivate and allow qualified people to contribute here. In that i very much disagree with @kocio-pl - making it simpler for people to make simple feature additions by reducing the scrutiny in reviews will not help us in the long term. And attracting people competent to work on the larger problems and the overall cartographic quality requires a clear direction so you can rely on a change working positively in that direction not failing to be accepted because we, the maintainers, are in disagreement about the fundamental goals and cartographic vision. |
|
@kocio-pl - we've now had another month and a half since last discussing this. What are your thoughts now about #3951 (comment) and #3951 (comment) as well as @imagico's most recent comments? |
|
Since it seems this discussion is not going to make any progress i wanted to make a different suggestion. For that let me first characterize what i perceive to be the main problem of the current situation. This issue was opened by @jeisenbe after we had re-established the consensus principle for design decisions in this style about a year ago while practically for 1-2 years before changes were usually merged without consensus. During this time this style moved significantly away from a consensus position that all active maintainers could support. Also many of the major problems and strategic questions were not addressed during that period while development aggrevated the need to address these issues. But now that we have decided to make decisions based on consensus again the need to get this style into a state that actually represents consensus is imperative for the project to function and fulfil its purpose. And it has become clear i think that this is currently not working because the status quo is at least by some apparently deemed more desirable than any potential direction towards consensus. I think that is well illustrated by the discussion on various issues and PRs where consensus is currently missing. Since consensus based decisions - as mentioned right at the beginning of this issue - cannot work when people veto changes without offering a viable alternative solution or a very good reason for why the change is not necessary/desirable i propose the following changes to our decision making procedure:
Now we almost certainly will not get unanimity among maintainers for this but my suggestion would be that if we can achieve rough consensus under rule 1 for this we implement this change in procedure - assuming @gravitystorm as owner of the project is ok with that of course. An alternative approach if the above seems a too radical or too drastic step to some of us would be to create a new branch where these rules apply that would not be initially rolled out on osm.org but it would be developed with the aim to replace the master branch in the future. Some development rendering server with a global database would be important for this to work of course. This way we could test the new rules for viability first and if this decision making model allows achieving a consensus style again and turns out to produce better results we can merge it into the master. This would of course essentially mean to feature freeze the master branch of this style - which is however de facto already the case right now in many aspects. Would be interesting to hear what others think of these ideas. Alternative propositions would be welcome as well of course - but they would need to offer a viable way that allows this project to fulfill its function and pursue its goals in the future. I hope these considerations also illustrate how serious our current inability to make consensus decisions and to agree on a common direction for the project is. I do not suggest such radical steps lightly. |
One issue is determining this. Some other systems have a facilitator or chairperson who "makes the call" that rough consensus has been achieved, but I don't think @gravitystorm wishes to be involved too frequently in settling such debates. An alternative would be that a majority or super-majority of the maintainers would need to specifically say that they believe rough consensus was achieved. |
|
Re: forking the style. The vector tiles idea is close to requiring a new fork to be created, to make the further changes that are necessary for server-side vector tiles, but which would not really be beneficial for this style with the current tool chain. One of the motivations for designing a vector-based rendering would be to provide client-side vector tiles as an alternative option at openstreetmap.org, which would potentially make it possible to show more features relevant only at high zoom levels in cities. So it would make sense that the vector fork would be the place to continue some of the urban-mapping centered ideas that were used last year. |
Yes, that would be an option. If that is preferred we could also elect a facilitator - but i would be somewhat reluctant to rely on a single person's judgement w.r.t. design decisions. It is my hope that as a group we have the ability to make responsible decisions moving the style towards consensus as long as we have a robust enough procedure that is able to deal with not everyone being constructive at all times.
First of all what i suggested is not a real fork of this style but an internal branch developed under exactly the same governing framework as the master branch except for slightly different decision making rules. This would facilitate the intended integration back into the main branch. Any branch that breaks design continuity to osm-carto from the start - and i don't see how any vector tiles fork could work without doing that - should IMO be a real fork - with a new team of maintainers and choosing new governing rules. Not doing so and not taking the opportunity to start off with a clear cartographic vision and clear goals in the vague hope that administrative continuity to osm-carto would either facilitate developer recruitment or being accepted for deployment on OSMF infrastructure would IMO be short sighted. But i don't want this to sidestep the important discussion here - until someone actually starts a concrete project and concrete implementation with a vector tiles fork this would be what in German we call "über ungelegte Eier reden". |
That's the direction it should go IMO. Otherwise, a single person could just stall forever on making a decision. Causing things they don't like to not get implemented but also not be rejected either. Which shouldn't happen. Having a facilitator or chairperson wouldn't work because it puts to much burden of decision making on a single entity. There isn't anyone that committed to or knowledgeable about the project at this point anyway, but even if there was I don't think its a good direction to go in. On the vector tile thing, a new development team for it would probably be good. But also with a close watch and guidance from this one until it gets off the ground. As I think they will probably be more similar projects then not and a lot of the same guiding cartographic visions here would apply there. Plus, there's some important design discussions from here that could and should transfer over to there. Also, I agree there should be some sort of cartographic standards (vision) in place at the start of the project. Implementing vector tiles shouldn't be held off or stalled until they are 100% perfect or worked out though. So, nothing like "well, we don't know what specific icons should be on z17. So, no vector tiles until we decide." There's no reason vector tiles can't be rolled out at the same time the details of the cartographic vision for it are being ironed out. Otherwise, vector tiles will just be DOA. |
|
Note that the use of majority/super-majority brought up by @jeisenbe was not meant in replacement of consensus decisions, it was meant to facilitate determining rough consensus as described. The difference between a super-majority and rough consensus is that the latter still requires any opposing voice against a decision to be considered and discussed. Only if the opposition is non-constructive and does not offer a viable way forward it would be disregarded. A super-majority on the other hand would allow disregarding small minorities even with the most weighty arguments. This is not a good idea IMO because it incentivizes lobbying and negotiation as basis of the decision making process rather than arguments and reason. I have shown in #3969 (review) how such a decision could be supported. |
|
One note from my engineering experience. Having general guidelines or goal definitions helps a lot in several ways:
I'm not saying it's a panacea, but it could help to a visible degree. Finding consensus is not easy, finding it over and over again is way harder. Quick example. There's #4138 , where some color-coding is discussed. If we had it written somewhere that when possible, we prioritize international conventions over national ones, half of the comments wouldn't have happened as they are because they simply reference the local standard without any argument explaining why this local standard is better than another local one. |
Although I mostly agree with your comment in principle, IMO what we should go with is what's "best" for the particular thing being rendered and there might be cases where that's a local or national standard. So, those things shouldn't just be discounted off hand. I doubt most people viewing the map would know or care what regional level the rendering is being based on anyway. The important thing is that the rendering makes intuitive sense or not. Which I don't think is necessarily dependent on where the standard comes from. Also, OSM is already inherently nationally and locally based when it comes to most things having to do with it. For instance tags are based on British English. Which is probably for the better, because there is no such thing as a "national" language. Let alone a "national" form of English. Plus, there is just an inherent European male bias to the demographics of OSM's contributors. Especially in the higher echelons and in who makes the decisions. Not that there is anything wrong with that, but it does make it more likely that things will be implemented based on local or national standards and conventions, instead of global ones. Although, it's deeply flawed to think there is a "global" standard for anything anyway. Even with things like ISO standards. |
|
@Adamant36, just a remark. The quote above is an example of what I meant as a possible guideline/goal. Not a suggestion within the scope of this discussion. (While it is a suggestion within the scope of #4138). So, there was no need to address it specifically here. |
|
I know. I was just giving my thoughts on it. What I said is relevant to the decision process. Maybe there wasn't a "need" for me to comment about it, but last time I checked this is a discussion and I'm allowed to comment on what I feel like 😉 |
|
So with this issue still open, how are decisions currently made? |
|
Decisions are currently made through consensus of the maintainers. The problem - and what this issue is about - is that we currently can't achieve consensus on a significant number of concrete decisions as well as on the overall strategy in design as well as guiding principles for development work. See
consensus needed
There is a rather limited set of changes, in particular bug fixes and smaller adjustments as well as purely technical changes without significant visible impact, where consensus can be achieved. The only changes made during the past year that went somewhat beyond the simple maintenance level were #4168 and #4218. |
|
I understand that this deadlock in the decision-making process is a persistent issue. What kind of actions do you take when you can't reach a consensus among the maintainters? Do you involve other people in the OSM community to assist you in the decision-making process? |
|
First of all:This is an open project, the deliberation of the maintainers on decisions is public (with rare exceptions, in particular on the nomination of new maintainers). You can see how decision making works (or does not) by following this issue tracker. And there are many not actively involved in style development who do so and provide input on occasion when it seems useful to them. Hence the question if we involve other people in the OSM community only makes limited sense - we do so constantly in many ways. Also keep in mind that - as said many times - this is a map design project and not a software development project. It is much harder to show beyond doubt that a certain decision is going to be beneficial or non-beneficial - even for decisions that are definitely decidable in that regard, i.e. where in retrospect you can objectively demonstrate them to be one or the other. That is a huge part of the problem. Many of the issues we currently lack consensus on are caused by past decision (in particular those having been made without consensus) leading to serious design issues but there being no agreement about the broader direction in which to move to resolve these things. We have above briefly discussed the option to choose a facilitator. IMO that could still be a useful idea - either in the role of enabling and supporting us to develop a common strategy and guiding principles for development work or as someone who is making the final decisions if there is no consensus to be found. The second option would of course mean a fundamental departure from the current consensus based decision making model. For either of these options finding a qualified person willing to volunteer their time for this would be hard. And of course it is a hen and egg problem (because we would need consensus to decide to do this in the first place) and a huge part of the problem is that some of the maintainers consider the status quo more desirable than any potential consensus on the future direction of this project. What we definitely will not do is turning design decisions here into a popularity contest among people who have no investment in the goals of this style and who are free to put their own short term interests above the long term sustainability of this project. Attracting competent developers and producing a good quality map will always require making also unpopular decisions. |
|
Maybe what could help is, if you give yourself a deadline by which you are going to solve this issue. At least there will be a movement towards the right direction... |
I'm pretty sure I suggested that once and it was ignored. In retrospect I don't think rendering decisions should be made based on artificial time constraints. It could lead to unnecessary mistakes and this is a volunteer project anyway. Also, as I'm pretty sure @imagico said this whole thing is pretty circular. There needs to be a consensus to figure out the lack of consensus. IMO opinion there are a few ways it can be resolved, but they either won't happen on this end. Like getting a facilitator or expanding the Collaborator pool so any single person has way less power to stall things and push their agenda, or would take broader intervention and investment by the OSM organization. Like implementing vector tiles or increasing the zoom level. Both of which would increase the amount of rendering flexibility. Although, even then at a bare minimum there would still have to be design goals. Which there isn't. So, the whole things circular or just takes multiple things to be implemented that rely on each other. |
|
@Adamant36 - to make one thing very clear: What is discussed here is a social issue. There are no technical solutions to this. No matter what certain proprietary software vendors in the domain might promise: There are not tools that magically produce good cartography. You need competent designers for that - working together in a consistent fashion. The list of issues i linked to earlier contains predominantly issues that are not subject to significant technical constraints. And even in those where technical innovations could help resolving the issue we have often determined that availability of such would not lead to consensus (like #3750 (comment)). In short - this is a problem that needs to be addressed on the social level. You are quite on point in one thing however: If we cannot solve this problem we will more and more fail to fulfill the function of maintainers of this style and this project will fail to fulfill its function in the OSM community. That would as you indicate over time and step by step probably lead to other projects replacing OSM-Carto in practical applications. Or as i put it a long time ago: This project is a social experiment, cooperative map design on this scale and with the ambition to produce a map for the global OSM community is something that has never been tried before. And it is something that can surely also fail. If it does depends on us and not on the technical circumstances we are confronted with. And the idea of nominating additional maintainers to make it easier to achieve and determine rough consensus has crossed my mind of course. But again - that would require consensus among the existing maintainers. And of course also qualified candidates for the position. @Mashin6 - declaring a deadline without specifying what happens if you don't meet it does not work of course. What we have tried is suggesting simplifying feature removal and rolling back changes made in the past without consensus in cases of lack of agreement between maintainers (#3977) - which is kind of similar to the idea: If we can't find agreement we default to removing those map elements that stand in the way of consensus in map design. It does not specify a deadline but as @Adamant36 already indicated that would be problematic in a volunteer project anyway. It would however create a robust way to get movement towards consensus again. Unfortunately however as you can see there is no consensus so far on that - largely because it would require abandoning the safety of the status quo in the hope there is consensus to be found that is better than the status quo. I am convinced this is the case and one thing i have tried over the past years is demonstrating in various examples (see here, also explained in #2270 (comment), various more detailed discussions here) is what you can achieve if your consistently follow an overall design paradigm. In other words: Resolving this would require everyone to abandon the glass half empty view in favor of a glass half full attitude. |
|
Sure, one can bring in some pressure from the community like the one you mentioned, with phasing off of the project in favor of a different one. But I would still assume that maintainers signed up for this, because they are motivated to create a great map. And if it is so then they should be able to set their time goals themselves. |
|
@501Ghost Thanks for the insightful question. Clearly aknowledging the community input on decision making sounds like essential long-term policy towards healthy project. It would help to work against constant drift of OSM Carto into de facto private map "fork" when the feedback loop is very loose. Just the fact, that community can read and write, does not work in practice, if most of the time maintainers spend on rejecting or ignoring them. That's just kind of open discussion forum related to the code, not the real community project. |
Follow-up to #2436
Define the method of making decisions based on consensus
Proposal: we should agree to a process for consensus-based decision making and document this in
CONTRIBUTING.md,CARTOGRAPHY.mdor another appropriate location.As @gravitystorm has stated, we should:
"Build consensus and agreement through discussion and compromise, using a combination of the cartography guidelines and common sense."
"Consensus doesn't mean unanimity, but there shouldn't be anything merged here against firm opposition from a maintainer. On the other hand, a maintainer shouldn't block a change without offering a very good reason, or another way forward. If there's opposite opinions on something that's not covered by the guidelines, then we can add more guidelines or update the existing ones."
We should discuss how this should work and how to resolve cases when two maintainers disagree.
We should also discuss whether a review by any contributor should be given equal weight, which would mean that all objections to a proposed change would need to be treated equally. This would also increase the need to define when consensus has or has not been achieved.
The text was updated successfully, but these errors were encountered: