[RFC 0136] A plan to stabilize the new CLI and Flakes incrementally #136
Conversation
Ericson2314
changed the title
Can just refer to NixOS#136 now.
|
Hm. I think the issue with Flakes is not just that they become required to use some feature X, but that they break subfeature X.Y of X in the process… |
rfcs/0136-stabilize-incrementally.md
Outdated
|
|
||
| ## Step 2: Stabilize the store-only Nix CLI | ||
|
|
||
| If the deadline for step 1 is met, then we stabilize *just* the command-line interface of the store-only Nix command. |
There was a problem hiding this comment.
Actually, even if store-only Nix is not there, can nix command be stabilised area by area, probably with store-related commands first?
There was a problem hiding this comment.
Yes, but I worry we will accidentally mess things up without the rigorous correct-by-construction layering that building subsets of Nix gives us.
Basically, in my opinion proven freedom from layer violations is a major way we can tell a "plumbing" command (to use the git term) is good.
There was a problem hiding this comment.
Actually current RFC #134 does not seem to require that eval-part of Nix goes only through the CLI-exposed part of store-Nix, does it?
There was a problem hiding this comment.
No it does not, but I meant making sure low level commands don't need high level library code.
There was a problem hiding this comment.
But this doesn't really prove that the store-Nix is useful as plumbing without layering violations (like the current way of specifying a specific output), does it?
There was a problem hiding this comment.
Oh OK, I thought you were saying we needed a more stringent process, but it sounds like you are saying the opposite. Sure, that makes sense.
There was a problem hiding this comment.
I want a more stringent process but I also admit we don't need it as a hard requirement! (And a more stringent process has more cost)
There was a problem hiding this comment.
Sure I agree. I am hoping I get that prev RFC approved / PR done regardless and it's a non-issue, but if I fail then yes it is good that we can still stabilize things incrementally.
There was a problem hiding this comment.
@7c6f434c It seems you're on the same page now. Maybe you can mark the conversation as resolved? That would make it easier to see if there's still something left to discuss or not.
There was a problem hiding this comment.
I am not sure we were on the same page back then, but the situation has shifted enough to make the minor differences moot.
… as of the literal question you ask — as far as I can see, I cannot. GitHub probably doesn't think reviewers should be able to resolve the points they raise.
|
@7c6f434c the "encroaching on other issues" section was supposed to cover that. I personally didn't mind this a lot as I wasn't using many of the new CLI commands with Nix 2.3, but most people mention this to me as they main gripe with Flakes, so I was sure to emphasize it over my own more niche concerns. |
However, it doesn't cover it. It says «using the same features now requires converting workflows to Flakes». This issue is much easier to brush off than «using the same feature is now impossible because Flakes supersede the older and more flexible implementation». |
rfcs/0136-stabilize-incrementally.md
Outdated
| Flakes is very popular among these new users. | ||
| Ergo, Flakes is very popular among the Nix community as a whole. | ||
|
|
||
| Like it or not, these users have been using Flakes as if it was stable, and we cannot make huge drastic changes that would break their code in hard-to-fix ways. |
There was a problem hiding this comment.
This is the crux of the issue, and I can't agree with it.
Going into an effort to stabilise Flakes with the understanding that we can't actually change it very much would make a mockery of the RFC process as a whole. It would establish a precedent that following the RFC process is optional, and that if you don't want to follow it (and you're one of the privileged set of committers to the appropriate repository) you can work around it encouraging adoption of your "experimental" work until it gets to the point that "we can't make big changes" any more. Maybe the Flakes design is good enough that we wouldn't need to make huge drastic changes, but it still sets the same precedent if we go into it having accepted that we can't.
There was a problem hiding this comment.
One of the early breakages of the nix CLI was that it got hidden behind the --experimental-features flag. Exactly to address this concern and clarify that everything behind it is experimental. Even all the nix man pages say that it's experimental.
It's good that people are trying flakes because it provides feedback, but it seems pretty obvious to me that anything can be changed. There is no guarantee here.
There was a problem hiding this comment.
The 1 year Nix roadmap the NixOS foundation is working on contains an ominous reference to “start thinking about Flakes 2.0.” This will hopefully be clarified until NixCon, but it seems like breaking Flakes (in some controlled way, presumably) is already on the cards.
There was a problem hiding this comment.
I think Flakes 2.0 may not necessary be new features.
@alyssais I agree Flakes was advertised much more heavily than, say, Rust experimental features, which does make the whole process very dodgy. But I have also long since given up on trying to influence Flakes. @edolstra and others have a very specific vision on what UX they want for them, and I think a fine compromise is that they can basically do whatever they want with that so long as there remains a "rest of Nix" that is still fully functional and not an afterthought with huge gaps.
In other words, I would personally work very hard to make sure the first two round of stabilizations under the plan yield a good result, making requests for some small tweak dyed-in-the-whole Flakers find baffling or boring. But then for the 3rd round, I would largely bow out, having gotten the things I wanted.
Now, I am by no means demanding that persons besides myself do the same; there is still a 3rd round of stabilization for those interested in Flakes to voice their thoughts. But I gather a lot of the tensions around Flakes is predicated on a fear that Flakes will be the only way forward, with the non-Flakes functionality legacy stuff or otherwise left to rot. I think by making sure that the non-Flakes stuff is still good, or at least leaving room for that (e.g. we can rename nix search to nix flake search leaving room for the future re-emergence of Flake-agnostic searching) we can lower the temperature a lot in the room not only for people like me, but also for people that still care about Flakes being good (in their eyes) but can rest easier with there being a "plan B".
There was a problem hiding this comment.
I would say that pushing flakes through an imitation RFC process without properly addressing the criticism would damage RFC process without really adding anything to flakes (hard to get extra credibility from the process that just lost its credibility).
I guess with incremental stabilisation of Flakes we could reach a state where some things can neither be stabilised via an RFC nor removed via an RFC? I don't like this outcome too much, but maybe this would at least reduce the splash damage…
(From experience, I am pessimistic about making consistent good decisions about issues, but somewhat optimistic about compartmentalising the damage)
There was a problem hiding this comment.
@7c6f434c The common goal would be stabilizing the flakes-agnostic CLI.
There was a problem hiding this comment.
Even there, I think some people want nix profile to work (without flakes; but also whether it working at all is good or bad is likely to have less than 100% agreement)
There was a problem hiding this comment.
@alyssais I agree. Saying "flakes are widely used, so we can't break interfaces even if they're not declared stable" is flawed reasoning. @tomberek recently pointed out to me that leading figures have been encouraging users to adopt flakes since NixCon 2019. And it still happens in side channels all the time, new users being introduced with literally saying "it's fine, just enable experimental features, it's the right way to do things" or "it's strongly encouraged" – by whom?
These are people's opinions, they are free to have them, and others are free to believe them, but the manual and the guards in the code clearly state: this interface is subject to change.
Don't rely on experimental features in production, unless you know what you're doing and know how to get yourself out of a mess. There is no deadline on stabilising the new CLI or flakes, because we don't want to get stuck with what we may end up regretting.
I'm not in the position to conclusively judge and not willing to opine whether the new CLI design or the flakes paradigm of organising Nix projects are the right way forward and whether the improvements in user experience are worth the sunk cost. But I'm confident with stating that the kind of signaling and narrative we have observed
- Confuses beginners, produces uncertainty, and associates these negative emotions with the project, which tend to stick due to the primacy effect
- Reinforces distrust in Nix maintainers among new users, who we can expect soon to be the overwhelming majority
- Exerts needless constraints and pressure on maintainers where the whole point of experimental features is, well, to be able to experiment, and therefore having the freedom to make breaking changes.
There was a problem hiding this comment.
This sentence is now removed!
There was a problem hiding this comment.
And it still happens in side channels all the time, new users being introduced with literally saying "it's fine, just enable experimental features, it's the right way to do things" or "it's strongly encouraged" – by whom?
I won't even read non-flake tutorials I must say because it feels like the old way of doing things.
I'd say, why not both? It's fine to point new people to flakes if that's easier to get them onboarded (which many people believe it is) and even then it's fine to massively break flakes in the future if you get the design right.
|
I'm grateful to you for starting this conversation. I too believe that an incremental approach to stabilisation is the only way we'll get out of the situation we're currently stuck in, and it will be a big relief to have a well-defined plan for it that has been the subject of community input. Even if I don't like the plan we end up with, it'll be a better situation than what we have today. |
|
Is Step one (#134) absolutely necessary for Step 2 and Step 3? |
It is explicitly written that it is not absolutely necessary for step 3, and we seem to approach declaring it not absolutely needed for step 2. However, if it can be done in reasonable time, it might improve the quality of step 2. |
Thanks! Co-authored-by: sternenseemann <[email protected]>
|
Thanks for fleshing this out (and sorry for taking that long to answer). I agree with the broad idea of stabilizing stuff incrementally. There's a huge pile of experimental stuff, ranging from “experimental on the paper, but too widely used to be changed” to “actually experimental, we might drop it tomorrow”. The problem that makes this whole thing much more complex is that Flakes and the new CLI are strongly tied together:
All that being said, I think I'd favour trying to stabilize Flakes first for consistency reasons: A Nix with Flakes but not the new CLI would be a Nix with a not-so-obviously-good feature hidden somewhere within the language (under |
|
@thufschmitt I completely agree that some CLI commands are tied to flakes but others are not --- or only indirectly via flake "installables" syntax. I still don't see any reason we cannot stabilize the parts that are not directly tied to Flakes first.
OK, so the CLI without Flakes will not have
I don't think it is hard to stabilize
We'd be just stabilizing the plumbing commands. Blind spots are completely acceptable and expected. I am guessing this is a simple understanding where you thought by "stabilize flakes" I meant just the file format; I mean the file format and the CLI commands that are tied to it. They go together, and the value of one does depend on the other. Trying to stabilize those in isolation would mean we are unable to judge the merits of either properly. I do think some of those Flakes commands (like |
| feature: stabilize-incrementally | ||
| start-date: 2022-09-15 | ||
| author: John Ericson | ||
| co-authors: (find a buddy later to help out with the RFC) |
There was a problem hiding this comment.
Is this still supposed to say “(find a buddy later to help out with the RFC)” even though it’s been merged?
This typo got added at the last minute while other mistakes were being corrected. See [1] and [2]. [1]: <NixOS#136 (comment)> [2]: <NixOS@64fe8f3>
This typo got added at the last minute while other mistakes were being corrected. See [1] and [2]. [1]: <#136 (comment)> [2]: <64fe8f3>
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
Can someone update the status label? |
infinisil
added
status: accepted
and removed
status: FCP
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/2023-08-25-nix-team-meeting-minutes-82/32283/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/nixcon-governance-workshop/32705/9 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/stabilising-the-new-nix-command-line-interface/35531/1 |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/contribute-to-the-nix-2023-recap/37484/13 |
* nix-store-layer: Copy template * nix-store-layer: First draft * nix-store-layer: pluralism -> marketplace of ideas Thanks @fricklerhandwerk for the suggestion. * nix-store-layer: Tweak Tvix and go-nix section * nix-store-layer: Make Store team future work * nix-store-layer: Start tidying Guix section * nix-store-layer: Boild down Guix section further Implementation discussion is unneeded for now. * nix-store-layer: Make incrementality of design clear Do this by putting the steps in order with numbers, and showing how the hardest part can come last. Thanks @fricklerhandwerk for the suggestion. * nix-store-layer: Give RFC number * nix-store-layer: Fix typo Meaning was reversed! * Update rfcs/0134-nix-store-layer.md Thanks! Co-authored-by: Tor Bjornrud <[email protected]> * nix-store-layer: Fix typo Thanks! Co-authored-by: Adam Joseph <[email protected]> * Apply suggestions from code review Thanks so much!!! Co-authored-by: Valentin Gagarin <[email protected]> * Apply suggestions from code review Thanks @fricklerhandwerk, just tweaked a few things from your suggestions to make these. * nix-store-layer: Cut stabilization in future work down Can just refer to NixOS#136 now. * Apply suggestions from code review Co-authored-by: Linus Heckemann <[email protected]> Co-authored-by: Valentin Gagarin <[email protected]> * 134: We have a shepherd team! Co-authored-by: Linus Heckemann <[email protected]> * 134: Move manual and store-specific tests to future work As discussed in today's meeting. * 134: Alt name moved to future work --------- Co-authored-by: Tor Bjornrud <[email protected]> Co-authored-by: Adam Joseph <[email protected]> Co-authored-by: Valentin Gagarin <[email protected]> Co-authored-by: Linus Heckemann <[email protected]>
…ixOS#136) * stabilize-incrementally: Copy template * stabilize-incrementally: First Draft * stabilize-incrementally: Move RFC into position * stabilize-incrementally: Fix typo Thanks @0x4A6F! * Apply suggestions from code review * stabilize-incrementally: Fix typo! Thanks! Co-authored-by: sternenseemann <[email protected]> * stabilize-incrementally: There are easy thing we can do do first! * stabilize-incrementally: Word-smith * 136: Apply suggestions from code review Thanks! Co-authored-by: asymmetric <[email protected]> * 0136: Fix typo Thanks! Co-authored-by: Andreas Rammhold <[email protected]> * 136: Fix typo * Add shepherds * 136: Make clear we are not stablizing as-is * 136: Attempt second split * 136: Fix typo Added a word by mistake * 136: Explicate that stabilizing Flakes requires future RFCs * 136: Clean up relationship to 134 - Make clear it is accepted - Make clear the implementation on it is well underway, and PRs have already been merged. - Remove language about deadline and skipping steps if deadline is not met, as it is confusing and hopefully not necessary. - Instead include more open-ended language about reconsidering the dependencies if unexpected delays with the split do arise. * 136: Clarify drawback more, fix typos * 136: Beef up detailed design with reusable process * 136: Apply suggestions from code review Thanks! Co-authored-by: Valentin Gagarin <[email protected]> * 136: A few misc things * 136: Remove "no-Flakes Nix" from future work It is now an optional step in the RFC proper. * 136: Remove the infamous "like it or not..." By popular demand! * 136: Retitle some stesp for clarity * 136: Slight reword * 136: Layering principles and other changes from meeting today Collaborative editting is nice! * 136: Reword first sentence * More collaborative drafting * Update summary * More updates from collaborative editing in meeting * Typos, sort of address TODO * Fix numerious typos and small issues Thanks so much! Co-authored-by: Valentin Gagarin <[email protected]> Co-authored-by: Felix Uhl <[email protected]> * Remove last mention of a deadline The rest of these were already removed * Fix typo Thanks! Co-authored-by: Ryan Lahfa <[email protected]> * fix: minor fixes --------- Co-authored-by: sternenseemann <[email protected]> Co-authored-by: asymmetric <[email protected]> Co-authored-by: Andreas Rammhold <[email protected]> Co-authored-by: Eelco Dolstra <[email protected]> Co-authored-by: Valentin Gagarin <[email protected]> Co-authored-by: Felix Uhl <[email protected]> Co-authored-by: Ryan Lahfa <[email protected]> Co-authored-by: Tom Bereknyei <[email protected]>
This typo got added at the last minute while other mistakes were being corrected. See [1] and [2]. [1]: <NixOS#136 (comment)> [2]: <NixOS@64fe8f3>
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: |
Ever since the closing of RFC 49, we've had the new CLI and Flakes marked as experimental, with no clear plan forward.
With the goal of ending this current limbo and soothe longstanding tensions in the Nix community, this RFC does two things:
Establish general principles about Nix's architecture and evolution in order to ensure we do not get in this situation again.
Notably we are allowed to make breaking changes to experimental features, which includes both the new CLI and Flakes, until they are stable.
Establish an incremental plan adhering to the principles deciding on the order and priority in which to stabilize these features:
First, the non-Flakes CLI wll be stabilized, in phases.
Afterwards, Flakes itself and its CLI components can be stabilized. The final design of Flakes will also require another RFC.
Rendered
A note to anyone leaving comments, remember the point of this RFC is not to debate the content of what is being stabilized, but debate the process of stabilization. To my fellow Flakes skeptics in particular, please hold back on the minutiae what you think is wrong with the design of Flakes, the new CLI or other features, or we will end up with another monster RFC thread that goes nowhere.
The current trajectory as far as I can tell, prior to writing this. is that we all of the sudden get a Nix 3.0 where a huge swath of stuff without review becomes stable at once. This is because many people in charge feel exhausted, and fear properly audit and debate the new features will devolve into yet another messy useless flame war.
Only by keeping the conversation narrowly on topic can demonstrate that a cordial compromise is possible