Auto-add of untracked files screws me up every time #323
Comments
|
Agreed on that — I think automatically committing tracked files is fine, but untracked is probably bad:
|
|
I mostly find the feature useful, but I also agree that it can be annoying and confusing. The worst case I've noticed is when you - perhaps accidentally - check out the root commit, where there's no When looking at an old version of the repo, you'll not see untracked files (e.g. |
I plan to use this matcher for some future `jj add` command (for #323). The idea is that we'll do a path-restricted walk of the working copy based on the intersection of the sparse patterns and any patterns specified by the user. However, I think it will be useful before that, for @arxanas's fsmonitor feature (#362).
I plan to use this matcher for some future `jj add` command (for #323). The idea is that we'll do a path-restricted walk of the working copy based on the intersection of the sparse patterns and any patterns specified by the user. However, I think it will be useful before that, for @arxanas's fsmonitor feature (#362).
|
Just to throw my 2c in: With the exception of new files that are also added to |
|
Just wanted to say that I really like the auto-tracking feature as well. After using jj for a while, going back to manually having to think about adding files to be tracked seems like a lot of extra work and possibly more error prone. I'm already used to checking |
|
The thing with accidentally committing GBs of Especially if you have something automatically creating it, like The only way is to recreate the repo (losing the oplog) this time not forgetting to ignore stuff/not editing old commits - which is kind of meh as well. Full GC of course means basically the same thing, only automated by a single command, but something like |
|
Maybe it is possible to track ignores in P.S. Can think of how |
|
Hm, so auto-tracking when you are doing changes in a working commit is the main killer feature. But for my issue with it, how about this: I know this is kind of magical, but the more I think about it the more it makes sense, idk - and the autorebase is magical on it's own, this somehow is kind of even consistent in my head |
|
It would be expensive to find the gitignores from all commits, but we could probably index that information. I'm more concerned that it would be unexpected behavior. For example, if you check out a sibling commit where Maybe it's better to check if the gitignores changed between the old and the new commit and if any untracked files according to the new patterns match the old patterns. If that happen, we could just print a warning about it. We could additionally add the ignores to a per-workspace set of ignores (which we don't support yet). (EDIT: I think this is what @ony suggested.) |
The warning being "those differences are implicitly untracked for this WC, in case your direnv caused GBs of files to generate in target and/or .direnv - add them to gitignore here or explicitly track them, moving to another commit without them ignored (e.g. ^ this is a loose idea, could be refined, for example |
|
Following @ony's suggestion, perhaps when the working copy moves to a new commit, we could track "newly unignored" files by comparing the old Then, the UI could provide ways for dealing with these files. E.g. there could be a command like |
|
I ran into this today when trying to checkout to a different branch that doesn't contain |
|
Ahh ignore my comment, I think I got confused - once there is an option to not auto-add untrack files |
You can pass jj op log --ignore-working-copy # "jj op log" also works with the current main branch
jj op restore --ignore-working-copy @-
jj workspace update-stale --some-option-to-not-snapshot-before-resetting |
|
Overall, it seems there is no good automated way to handle untracked files when creating commits. One one hand, not tracking them leads to incomplete commits. On the other hand, auto-tracking them leads to commiting of unwanted files. So how would you feel about some variation of the following semi-automated design?
I think this might strike a good balance between the following concerns:
For scripted operation, there should be a way to provide a default answer to the prompt via CLI arguments. |
|
I'm personally quite happy with the current behavior (except for the behavior when updating to a commit with different .gitignore). It can be a bit annoying in the beginning, but once you've added the appropriate paths, I find that it works pretty well. Maybe others feel differently. But even if they don't, we may want to make it less annoying for new users by doing something like you suggest. |
|
I'm not very happy with the idea of the interactive prompt. I think that if you edit a Other UIs will also do an analogue of |
|
Ah, yes, there's that. I knew that this design decision of having status commands modify the repository was fishy and going to cause problems someday... |
|
This feature has sadly made me bounce off of jj immediately every time I try it, which is really unfortunate because I keep hearing such good things about it, and want to give it a genuine try. Every single repository I work on regularly has various testing/strace-log/whatever files in its root. I actually don't mind auto tracking in fwiw a workaround for this that I've not yet checked works on jj might be some kind of terrible thing like so in the This workaround is quite bad indeed, and I would rather not have to reimplement the git index in |
|
I have no idea whether this would be helpful you, but here's something that helped me a lot. I can't remember who had suggested it originally; it might be in the FAQ.
|
|
Yup it is in the FAQ or something; I've seen it given as advice before. I just don't like it and it doesn't vibe with how I work, since it would be a whole bunch of extra typing. I could have it be i/ or something, I guess, to reduce typing, but I would still have to remember to do it every time, which feels kind of bad? |
|
Inspired by more feedback (#3528 (comment)), perhaps @dpc 's suggestion from that post might work. Perhaps we could have a notion of "untracked" files, like Git, and default files to "untracked", while also auto-updating all the tracked files on each command?
One question is what This would be a huge change, so I almost certainly missed some important considerations. |
Speaking out of ignorance, I'm guessing Hmm... I guess |
If we add support for untracked files, I think it should be pretty much only
I don't think so. Almost all commands, and probably also the future |
This is tricky... also abandoning commit that adjusted .gitignore. Another problem - .gitignore file itself might have merge conflict markers. Here is a different direction: make ignored files/directories "first class" citizens and maintain as a property of the working copy. In this case Would something like that be a reasonable behaviour? |
I discussed this a bit in my "train of thought" comment, see also below.
I think that's a separate issue that I'll ignore the rest of @sheremetyev 's interesting comment for now (need to think about it more) and give a potential outline of a design for the "automatically update gitignore for newly untracked files" idea. I think it makes sense and might be viable, though I'm not sure it's the best option. All the details (esp. the names) are preliminary. We could give almost every
For most commands including If you are in that situation, and you wanted to get rid of both the For a few commands, e.g. (Strictly speaking, if we considered snapshotting an "operation", it would follow the "track" behavior) We'd also want detailed warnings if the As you can see, this is complex. I'm not sure we can explain it simply; we'd have to try. I'm also not sure any other design we choose, once we work out its details, would be simpler. These behaviors could occasionally be annoying, but mostly (hopefully only) in the cases where Implementing this might take some work, but the main question in my mind is how frustrating and confusing (or clear and painless) this would seem to users that run into these problems unexpectedly. Whatever design we choose, the issue of un-ignored files is almost always going to be an unexpected obstacle that distracts the user from what they are actually trying to do. |
|
Agreed @ilyagr! With all the edge cases automatically modifying Hopefully "first-class ignores" is simpler? I tried to think through various cases and don't see major issues. Short way to describe the behaviour:
|
|
iirc, there's another feature request to add some overlay that contains "volatile" changes (e.g. debug prints, environment-specific modification, etc.) that aren't meant to be included in the final commit, but should be carried over to new checkout. If we had such layer, newly un-ignored files could be snapshotted to that layer, not to the working-copy commit. Just an idea. It wouldn't solve the big "un-ignored node_modules problem" if the overlay is backed by a commit object. |
|
What if the working copy contained a list of prefixes of untracked/ignored paths (meaning any paths that aren't included in the The idea is that This would mean that if the user switches to a new commit without a .gitignore, the ignored files remain untracked since the paths are still marked as untracked in the working copy. It could also be possible to turn the file size limit error into a warning, and just add it as an ignored path in the working copy to prevent snapshotting the file until the user explicitly tracks it. This would probably also require a new section in This sounds similar to what @ilyagr was talking about in #323 (comment). |
|
The user should be able to reason easily about whether the file is tracked or not, but it seems to be getting too complicated to explain. |
|
@marc-h38 said:
Some parties in this conversation consider it a broken window which enables a "bad workflow" or a workflow a new vcs shouldn't have, while the other party really depends on that behavior. I think my opinion on that is known, so I've said enough. My opinion on the other conversation is that we're actually converging on a solution which @joyously brought up here: #4338 (comment), which definitely can work. Ideally this could be just a single binary state instead of the trinary behavior with the snapshotting option, but we have it now and must work with it. |
|
There's a lot going on in this thread now. (Good!) I'll just reply to a few things, but I'm still actively thinking. This will be inconclusive. I did, in fact, try to write down a conclusion or at least a summary, but I just don't have one in mind at this point. Re @sheremetyev
I haven't given up on it at all, I think it's very much worth considering. It seems likely to me that the reason it seems more complicated than the other ideas is merely that I explained it in more detail.
I am not sure about treating I don't think this is necessarily fatal to the idea. For example, after editing .gitignore, some files could go into a weird middle state and @scott2000 's idea seems very close (or very compatible) with this one.
I didn't follow this part. Re @yuja
Good point! I think this is worth considering as well. From a certain perspective, it's very similar to what I was suggesting in #323 (comment). From another perspective, this overlay could be a way to implement something like Fedor's idea or even some extension of
If this is a problem, we could try to keep it in an easily garbage-collectible commit, though that might come at the price of not having This does come at some cost to usability for the "scratch file" usecase, but maybe that's OK? We don't have to tell people there was ever an option of tracking their scratch files in the operation log. 😇 |
Re @joyously
Aside: For other people, different aspects of this approach are discussed in #4338 (comment). The main question we seem to be discussing at the moment is what to do when the user switches to a file with a different .gitignore and some of the files that were previously ignored are still in their working dir. I think this is more or less solvable, as @sheremetyev suggested in, #323 (comment), but when I tried to flesh it out, the result is complicated (but possibly not too complicated, I'm not sure): #323 (comment). To be clear, I'm still interested in this approach, and @PhilipMetzger 's comment suggests he is too. It's not clear to me yet whether it will work.
I didn't understand this idea. |
|
I realized that, for better or worse, @yuja 's "overlay commit" is pretty much equivalent to Git's staging area. We would use it for a slightly different purpose, though, it wouldn't be as front-and-center as in Git. |
What I meant was that the ignore file is a special case since it is a control file, and whenever it is modified, that should be separated from other changes into its own commit, which is then applied as far back in the history as possible so the user can't switch to before and after that ignore. As for the ignored files, a true ignore should not delete them, but not snapshot them either. |
I think that regardless of its merits, this idea (if I understood it correctly) is impractical for any repo where people are not allowed to amend a commit that's been shared with other people ("immutable commits" in jj parlance). This prohibition would include rebasing a commit on top of the new Perhaps once jj takes over the world and people can refer to commits by their change id, this will become possible to consider. Even then, there would be security implications and other assumptions people are currently used to being broken by it. |
Kind of? New working-copy changes would be squashed into the overlay's parent, so it's conceptually quite different from the staging area. Oh, but that means changes on un-ignored files wouldn't be ignored, so it's not what the user want. |
|
I am liking the editing .gitignore idea, especially after Yuya's point as I understand it (which deserves a longer reply/explanation I won't write right now). One moderate problem with the editing .gitignore, however, is how it deals with multiple workspaces. I wonder if anybody will have any ideas about that. I think that what I wrote in #323 (comment) is actually OK as written if we require the user to specify The other possible solution would be to use a workspace-local ignore file for these, as I pondered in #4338 (comment). However, that blocks solving the problem of unignored files on solving the problem of how to do per-workspace gitignores in jj. I don't know an obvious solution to that problem, and that second problem is not otherwise as urgent, so blocking the original problem on it is also not great. Aside: In principle, we could also consider changing how |
|
I'm skeptical about auto-updating I think the idea described here can be a good compromise.
(FWIW, if watchman is enabled, the current |
I was thinking of keeping it simple and listing each newly unignored file individually at the end of the .gitignore. The "newly unignored file" logic would only even add new lines at the end when needed, it would never delete or edit them. (Though, the user could remove them en masse with |
You mean "newly unignored file or directory"? If it were only files, tons of |
@ilyagr, I think the mental model here is quite simple, and "jj status" is the all UI we need:
|
That may be a good thing IMHO. If I make a mistake while editing .gitignore files, I'd rather not have files automatically tracked or untracked - to not complicate cleanup of the mess after my mistake. |
|
I think another issue with modifying .gitignore is that it's unclear which ignore file should be modified. Often when I ignore files, I only want them to be ignored for me and I don't want to modify it for my coworkers as well (because they're just notes or some files specific to me), so probably 90% of the time they go into .git/info/exclude instead of .gitignore. I also occasionally might want to add it to my global .gitignore instead of the repo-specitic one. In the future, it also seems like it would be good to transition away from .gitignore and support .ignore as well, especially for the native backend. If we were to go with the approach of modifying .gitignore, I think the safest option would be to have a workspace-specific, local ignore file that automatically gets modified when you track/untrack files, since it feels safest not to automatically add a change to a commit (since you might not intend to share that change with coworkers). But then at that point, I feel like it's basically equivalent to what I was describing with adding a list of untracked files/paths to the working copy itself. |
It is an excellent point that we can consider this. This is such a big change to the UI that I didn't consider it seriously, but I agree that it might be a good thing. A lot of the complication of some of the ideas I described comes from trying to keep up with potential changes to .gitignore. We already have the concept of .gitignore and a separate list of tracked files. If we can make the usecases work with just commands to manipulate these, that'd be great, and I think there's a chance of a UI could uniformly treat different categories of tracked/untracked files.
This is currently my biggest worry about this idea -- I'm not sure this will work (and I'm not sure that I follow your thinking). For one, what if the new commit has a tracked file where the working copy had an ignored file? This situation would already be a problem today, but I think it'll become far more common with your setup. E.g. it would be potentially problematic if you start on the Update: This usecase would be problematic with other solutions as well. So, on second thought, I'm not sure whether this problem is worse for Fedor's approach than for any other. It might be worth thinking about anyway. It seems likely to me that, if we want the user to manually resolve cases when .gitignore does not match the actual ignore files (or a subset of them that is not marked as scratch files), we would be forced to create a |
|
I want to keep this reply brief to give some space to Fedor's idea, but if we do not go with that idea:
|
|
For @sheremetyev 's idea, I'm wonder what to do in co-located repos where Git will get the list of ignored files from .gitignore. It seems like, in a co-located repo, ignored file that's not in .gitignore is OK, but not-ignored file that is in .gitignore could be a problem. I guess |
I think for .gitignore, this definitely isn't ok actually because they have different semantics. If I ignore Either way, I personally still don't feel very comfortable with I do see the value of such a command, but I would at least prefer for it to be something the user has to ask for explicitly (e.g. |
There may be a negative pattern in sub directory's
It would probably become simpler if the working copy tracked ignored directory/file paths. If the whole directory was previously ignored, and if it is now un-ignored, record it as new ignore path. However, this means the working-copy tracks ignored paths, so I think it's easier to accumulate ignored paths instead of updating user-managed |
Yes, I think the rules are basically appended to the list from the parent and processed in reverse order, so later rules in a subdirectory or in an individual |
@ilyagr that's a great example! I think it's reasonable to expect such file to remain untracked - because it's explicitly listed in the working copy's list of untracked files? Switching back to the main branch would put working copy in a good state (ignored file is still on disk). Intermediate situation on gh-pages branch is somewhat similar to the case where a file was committed previously, then user decided to stop tracking it but wants to keep the file on disk. IIUC, to achieve such behaviour, the list of untracked files in the working copy should take precedence over the list of tracked files in the current commit - when determining tracked/untracked status for a file.
IMHO it's an important advantage of UX in Jujutsu that user can always switch to another commit - would be nice to preserve it |
|
Tip Summary of current state for anyone landing here If you use jj with git and want to not automatically track any files, since #4338 you can set this configuration: Then to see untracked files: To add an untracked file: (The work remaining for this IIUC is to not require a |
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [martinvonz/jj](https://github.com/martinvonz/jj) | minor | `v0.21.0` -> `v0.22.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>martinvonz/jj (martinvonz/jj)</summary> ### [`v0.22.0`](https://github.com/martinvonz/jj/releases/tag/v0.22.0) [Compare Source](jj-vcs/jj@v0.21.0...v0.22.0) ##### Breaking changes - Fixing [#​4239](jj-vcs/jj#4239) means the ordering of some messages have changed. - Invalid `ui.graph.style` configuration is now an error. - The builtin template `branch_list` has been renamed to `bookmark_list` as part of the `jj branch` deprecation. ##### Deprecations - `jj branch` has been deprecated in favor of `jj bookmark`. **Rationale:** Jujutsu's branches don't behave like Git branches, which a confused many newcomers, as they expected a similar behavior given the name. We've renamed them to "bookmarks" to match the actual behavior, as we think that describes them better, and they also behave similar to Mercurial's bookmarks. - `jj obslog` is now called `jj evolution-log`/`jj evolog`. `jj obslog` remains as an alias. - `jj unsquash` has been deprecated in favor of `jj squash` and `jj diffedit --restore-descendants`. **Rationale:** `jj squash` can be used in interactive mode to pull changes from one commit to another, including from a parent commit to a child commit. For fine-grained dependent diffs, such as when the parent and the child commits must successively modify the same location in a file, `jj diffedit --restore-descendants` can be used to set the parent commit to the desired content without altering the content of the child commit. - The `git.push-branch-prefix` config has been deprecated in favor of `git.push-bookmark-prefix`. - `conflict()` and `file()` revsets have been renamed to `conflicts()` and `files()` respectively. The old names are still around and will be removed in a future release. ##### New features - The new config option `snapshot.auto-track` lets you automatically track only the specified paths (all paths by default). Use the new `jj file track` command to manually tracks path that were not automatically tracked. There is no way to list untracked files yet. Use `git status` in a colocated workspace as a workaround. [#​323](jj-vcs/jj#323) - `jj fix` now allows fixing unchanged files with the `--include-unchanged-files` flag. This can be used to more easily introduce automatic formatting changes in a new commit separate from other changes. - `jj workspace add` now accepts a `--sparse-patterns=<MODE>` option, which allows control of the sparse patterns for a newly created workspace: `copy` (inherit from parent; default), `full` (full working copy), or `empty` (the empty working copy). - New command `jj workspace rename` that can rename the current workspace. - `jj op log` gained an option to include operation diffs. - `jj git clone` now accepts a `--remote <REMOTE NAME>` option, which allows to set a name for the remote instead of using the default `origin`. - `jj op undo` now reports information on the operation that has been undone. - `jj squash`: the `-k` flag can be used as a shorthand for `--keep-emptied`. - CommitId / ChangeId template types now support `.normal_hex()`. - `jj commit` and `jj describe` now accept `--author` option allowing to quickly change author of given commit. - `jj diffedit`, `jj abandon`, and `jj restore` now accept a `--restore-descendants` flag. When used, descendants of the edited or deleted commits will keep their original content. - `jj git fetch -b <remote-git-branch-name>` will now warn if the branch(es) can not be found in any of the specified/configured remotes. - `jj split` now lets the user select all changes in interactive mode. This may be used to keeping all changes into the first commit while keeping the current commit description for the second commit (the newly created empty one). - Author and committer names are now yellow by default. ##### Fixed bugs - Update working copy before reporting changes. This prevents errors during reporting from leaving the working copy in a stale state. - Fixed panic when parsing invalid conflict markers of a particular form. ([#​2611](jj-vcs/jj#2611)) - Editing a hidden commit now makes it visible. - The `present()` revset now suppresses missing working copy error. For example, `present(@​)` evaluates to `none()` if the current workspace has no working-copy commit. ##### Contributors Thanks to the people who made this release happen! - Austin Seipp ([@​thoughtpolice](https://github.com/thoughtpolice)) - Danny Hooper ([@​hooper](https://github.com/hooper)) - Emily Shaffer ([@​nasamuffin](https://github.com/nasamuffin)) - Essien Ita Essien ([@​essiene](https://github.com/essiene)) - Ethan Brierley ([@​eopb](https://github.com/eopb)) - Ilya Grigoriev ([@​ilyagr](https://github.com/ilyagr)) - Kevin Liao ([@​kevincliao](https://github.com/kevincliao)) - Lukas Wirth ([@​Veykril](https://github.com/Veykril)) - Martin von Zweigbergk ([@​martinvonz](https://github.com/martinvonz)) - Mateusz Mikuła ([@​mati865](https://github.com/mati865)) - mlcui ([@​mlcui-corp](https://github.com/mlcui-corp)) - Philip Metzger ([@​PhilipMetzger](https://github.com/PhilipMetzger)) - Samuel Tardieu ([@​samueltardieu](https://github.com/samueltardieu)) - Stephen Jennings ([@​jennings](https://github.com/jennings)) - Tyler Goffinet ([@​qubitz](https://github.com/qubitz)) - Vamsi Avula ([@​avamsi](https://github.com/avamsi)) - Yuya Nishihara ([@​yuja](https://github.com/yuja)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
|
Thanks to @martinvonz and all the discussers for working on this. I'm able to use jj now because of the new setting. |
and others
Description
Initially I thought the auto-add of files was a neat idea, but in practice I just leave untracked files in my repo all the time, and tools like
patch(1) assume they can drop.origor similar files in the WC without it being a problem. I think every time I've usedjjI've ended up getting grumpy at auto-adds and having to rip something out of a commit, sometimes after doing a push (when it was effectively emulatinghg importfor example).Steps to Reproduce the Problem
patch -p1 < some.diff(or similar)jj describe && jj close && jj git pushExpected Behavior
I (still) don't expect auto-adds, and it really surprises me every time, plus it's super frustrating to have to ignore every file spec I might create as a temporary scratch file or what have you.
Actual Behavior
As above.
Specifications
The text was updated successfully, but these errors were encountered: