Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jj rebase --skip-empty unexpectedly changes the commit graph #2760

Closed
afgomez opened this issue Dec 31, 2023 · 10 comments · Fixed by #2766
Closed

jj rebase --skip-empty unexpectedly changes the commit graph #2760

afgomez opened this issue Dec 31, 2023 · 10 comments · Fixed by #2766
Assignees

Comments

@afgomez
Copy link

afgomez commented Dec 31, 2023

Description

When running jj rebase --skip-empty the commit graph gets mangled if there are empty commits created as a result of the operation.

Steps to Reproduce the Problem

  1. Create a repo with some commits
$ jj init --git test-repo; cd test-repo
$ echo a > a; jj commit -m A
$ echo b > b; jj commit -m B
$ echo c > c; jj commit -m C
$ jj log
@  ovpukpso [email protected] 2023-12-31 09:50:57.000 +01:00 18755ef0
│  (empty) (no description set)
◉  uuxmnpmy [email protected] 2023-12-31 09:50:57.000 +01:00 61676efa
│  C
◉  yvvmnvxw [email protected] 2023-12-31 09:50:49.000 +01:00 4b9ba9d2
│  B
◉  lrkotskn [email protected] 2023-12-31 09:50:39.000 +01:00 161d8858
│  A
◉  zzzzzzzz root() 00000000
  1. Create a new commit on top of lrk, with the same changes as yvv
$ jj co lrk
$ echo b > b; jj commit -m 'Another B'
$ jj
@  ypxrttrn [email protected] 2023-12-31 09:53:12.000 +01:00 7321edf3
│  (empty) (no description set)
◉  lxknpytx [email protected] 2023-12-31 09:53:12.000 +01:00 eca4b830
│  Another B
│ ◉  uuxmnpmy [email protected] 2023-12-31 09:50:57.000 +01:00 61676efa
│ │  C
│ ◉  yvvmnvxw [email protected] 2023-12-31 09:50:49.000 +01:00 4b9ba9d2
├─╯  B
◉  lrkotskn [email protected] 2023-12-31 09:50:39.000 +01:00 161d8858
│  A
◉  zzzzzzzz root() 00000000
  1. Switch to the commit with description 'C' and rebase with --skip-empty on top of lxk (the commit described as 'Another B').
$ jj co uux
$ jj
@  xsylwmsl [email protected] 2023-12-31 09:54:09.000 +01:00 9b0f9b9f
│  (empty) (no description set)
◉  uuxmnpmy [email protected] 2023-12-31 09:50:57.000 +01:00 61676efa
│  C
◉  yvvmnvxw [email protected] 2023-12-31 09:50:49.000 +01:00 4b9ba9d2
│  B
│ ◉  lxknpytx [email protected] 2023-12-31 09:53:12.000 +01:00 eca4b830
├─╯  Another B
◉  lrkotskn [email protected] 2023-12-31 09:50:39.000 +01:00 161d8858
│  A
◉  zzzzzzzz root() 00000000

% jj rebase -d lxk --skip-empty
Rebased 3 commits
Working copy now at: uuxmnpmy 79d445d5 C
Parent commit      : lrkotskn 161d8858 A
Added 0 files, modified 0 files, removed 1 files # <<< Note that a file is being removed. 

# vvv Note that the working copy is at 'C', not in a new empty commit.
# My guess is that `--skip-empty` is removing `xsy`.
# Note as well that 'C' is not properly rebased on top of 'Another B'. It's
# on top of 'A'
~/code/test-repo% jj
@  uuxmnpmy [email protected] 2023-12-31 09:55:44.000 +01:00 79d445d5
│  C
│ ◉  lxknpytx [email protected] 2023-12-31 09:53:12.000 +01:00 eca4b830
├─╯  Another B
◉  lrkotskn [email protected] 2023-12-31 09:50:39.000 +01:00 161d8858
│  A
◉  zzzzzzzz root() 00000000

In comparison, this is what happens with a regular rebase:

~/code/test-repo% jj
@  xsylwmsl [email protected] 2023-12-31 09:54:09.000 +01:00 9b0f9b9f
│  (empty) (no description set)
◉  uuxmnpmy [email protected] 2023-12-31 09:50:57.000 +01:00 61676efa
│  C
◉  yvvmnvxw [email protected] 2023-12-31 09:50:49.000 +01:00 4b9ba9d2
│  B
│ ◉  lxknpytx [email protected] 2023-12-31 09:53:12.000 +01:00 eca4b830
├─╯  Another B
◉  lrkotskn [email protected] 2023-12-31 09:50:39.000 +01:00 161d8858
│  A
◉  zzzzzzzz root() 00000000

~/code/test-repo% jj rebase -d lx
Rebased 3 commits
Working copy now at: xsylwmsl e3fc4b16 (empty) (no description set)
Parent commit      : uuxmnpmy 1fb4f069 C

~/code/test-repo% jj
@  xsylwmsl [email protected] 2023-12-31 09:57:26.000 +01:00 e3fc4b16
│  (empty) (no description set)
◉  uuxmnpmy [email protected] 2023-12-31 09:57:26.000 +01:00 1fb4f069
│  C
◉  yvvmnvxw [email protected] 2023-12-31 09:57:26.000 +01:00 ffeb7970
│  (empty) B
◉  lxknpytx [email protected] 2023-12-31 09:53:12.000 +01:00 eca4b830
│  Another B
◉  lrkotskn [email protected] 2023-12-31 09:50:39.000 +01:00 161d8858
│  A
◉  zzzzzzzz root() 00000000

Expected Behavior

  • The commits get rebased on top of the specified destination, in a linear fashion like when doing a normal rebase
  • jj creates a new empty commit at the tip of the branch

Actual Behavior

  • The commits create a branched history.
  • jj keeps the current commit at the last commit of the branch.

Specifications

  • Platform: macOS
  • Version: 0.12.0
@yuja
Copy link
Contributor

yuja commented Dec 31, 2023

My guess is that --skip-empty is removing xsy.

Yeah, that's probably it (though I don't know why uuxmnpmy wasn't rebased. There may be two separate problems.)

The @ commit is included in the explicit rebase source, and it's empty. In this scenario, (hypothetical) --skip-emptied (drop revisions that become empty) option would work better. We can of course add special case to recreate the wc commit if it was previously empty, but I don't know if that's reasonable.

@martinvonz
Copy link
Member

Hmm, that's a weird bug. @matts1, do you have time to look into it? Also, @ilyagr, since you're working on refactoring this code, do you want us to not touch this until you're done refactoring? I'm not sure exactly which parts of it you're going to touch.

The @ commit is included in the explicit rebase source, and it's empty. In this scenario, (hypothetical) --skip-emptied (drop revisions that become empty) option would work better. We can of course add special case to recreate the wc commit if it was previously empty, but I don't know if that's reasonable.

I think jj rebase --skip-empty should behave as a regular jj rebase followed by abandoning of any empty commits in the set of new commits (except that we don't even create the intermediate commits and set predecessor pointers for obslog).

@ilyagr
Copy link
Contributor

ilyagr commented Jan 1, 2024

Interesting bug. Thanks for finding the cause, sounds very plausible.

I think the bug could also occur with --skip-emptied, though it wouldn't be as common. The working copy can become emptied by a rebase even if it wasn't empty.

I think that if we identified the cause correctly, the fix would be to add another condition to --skip-empty's logic: don't abandon commits if they are merge commit (as it is today) or if they are the working copy.

Update: It's unclear to me what the best behavior is if the working copy has children. We could either rebase the children into the parent or not. At first, we could do the latter, since it's simpler.

Also, @ilyagr, since you're working on refactoring this code, do you want us to not touch this until you're done refactoring?

This wouldn't interfere with what I'm doing at all, unless I'm missing something and it's actually a much more complicated fix. The test should be based on #2738, which I'll probably merge tomorrow. (UPDATE: It's merged now.) Thanks for checking!

@matts1 matts1 self-assigned this Jan 2, 2024
@matts1
Copy link
Contributor

matts1 commented Jan 2, 2024

Was able to repro, thanks for filing the bug.

@matts1
Copy link
Contributor

matts1 commented Jan 2, 2024

After rereading the comments, I should probably clarify some things.

My guess is that --skip-empty is removing xsy.

This was intentional. I'm not a big fan of rebase changing how it works depending on your working commit, but I can definitely see the use for it here. I'm far more concerned about the fact that the history remains branched. There's open questions here (eg. "It's unclear to me what the best behavior is if the working copy has children").

For now, I won't touch this aspect of the bug. Someone else is welcome to make a change if they want though.

I care far more about the rebase creating seperate branches, which is definitely not WAI. I'll be working on fixing that.

matts1 added a commit to matts1/jj that referenced this issue Jan 2, 2024
Fixes jj-vcs#2760


Given the tree:
```
A-B-C
 \
  B2
```
And the command `jj rebase -s B -d B2`

We were previously marking B as abandoned, despite the comment stating that we were marking it as being succeeded by B2. This resulted in a call to `rewrite(rewrites={}, abandoned={B})` instead of `rewrite(rewrites={B=>B2}, abandoned={})`, which then made the new parent of `C` into `A` instead of `B2`
matts1 added a commit to matts1/jj that referenced this issue Jan 2, 2024
Fixes jj-vcs#2760


Given the tree:
```
A-B-C
 \
  B2
```
And the command `jj rebase -s B -d B2`

We were previously marking B as abandoned, despite the comment stating that we were marking it as being succeeded by B2. This resulted in a call to `rewrite(rewrites={}, abandoned={B})` instead of `rewrite(rewrites={B=>B2}, abandoned={})`, which then made the new parent of `C` into `A` instead of `B2`
matts1 added a commit to matts1/jj that referenced this issue Jan 2, 2024
Fixes jj-vcs#2760


Given the tree:
```
A-B-C
 \
  B2
```
And the command `jj rebase -s B -d B2`

We were previously marking B as abandoned, despite the comment stating that we were marking it as being succeeded by B2. This resulted in a call to `rewrite(rewrites={}, abandoned={B})` instead of `rewrite(rewrites={B=>B2}, abandoned={})`, which then made the new parent of `C` into `A` instead of `B2`
@ilyagr
Copy link
Contributor

ilyagr commented Jan 2, 2024

#2738 is merged now, so it shouldn't be an obstacle for writing a test for #2766 anymore.

Thanks, Matt, for working on this!

@ilyagr
Copy link
Contributor

ilyagr commented Jan 2, 2024

Tangentially related, @matts1, do you remember why we never implemented --skip-emptied in addition to --skip-empty? This is not a call to action, I'm mostly just curious.

matts1 added a commit to matts1/jj that referenced this issue Jan 3, 2024
Fixes jj-vcs#2760


Given the tree:
```
A-B-C
 \
  B2
```
And the command `jj rebase -s B -d B2`

We were previously marking B as abandoned, despite the comment stating that we were marking it as being succeeded by B2. This resulted in a call to `rewrite(rewrites={}, abandoned={B})` instead of `rewrite(rewrites={B=>B2}, abandoned={})`, which then made the new parent of `C` into `A` instead of `B2`
@matts1
Copy link
Contributor

matts1 commented Jan 3, 2024

I felt that there was a few open questions still regarding API.

Here are three options I was considering:

  • Have two flags, --skip-emptied and --skip-empty
  • Have an enum flag --skip-empty=newly|all|never
  • Have a single flag --skip-empty which does all empty commits
  • Have a single flag --skip-empty which does newly empty commits

I implemented the third option, but in retrospect, I'd probably be happy changing it to the fourth option. I personally think option 1 adds too much complexity to the API.

@ilyagr
Copy link
Contributor

ilyagr commented Jan 3, 2024

I like option 1 and option 4. Option 2 is annoying since the user will have to specify something each time; I don't think clap provides a good way to allow both --skip-empty by itself and --skip-empty all.

Update: If we go with option 4, I mildly prefer still calling it --skip-emptied as @afgomez suggested below.

It's probably not worth the trouble, but even if our goal is option 4, we could have both options for a transition period and then deprecate the old one.

matts1 added a commit to matts1/jj that referenced this issue Jan 3, 2024
Fixes jj-vcs#2760


Given the tree:
```
A-B-C
 \
  B2
```
And the command `jj rebase -s B -d B2`

We were previously marking B as abandoned, despite the comment stating that we were marking it as being succeeded by B2. This resulted in a call to `rewrite(rewrites={}, abandoned={B})` instead of `rewrite(rewrites={B=>B2}, abandoned={})`, which then made the new parent of `C` into `A` instead of `B2`
@afgomez
Copy link
Author

afgomez commented Jan 3, 2024

  • Have a single flag --skip-empty which does newly empty commits

To me this sounds like what I would want the tool to do. Only skip the commits if they become empty. If for whatever reason I have created empty commits (Like... I don't know... I want to outline how the history might look like before implementing anything) I think they should be preserved.

It's worth considering renaming the flag to --skip-emptied then, to make it clear that it would not skip currently empty commits.

  • Have two flags, --skip-emptied and --skip-empty

I think in practice I would only use --skip-emptied, but I can see the use case for having both.

matts1 added a commit that referenced this issue Jan 4, 2024
Fixes #2760


Given the tree:
```
A-B-C
 \
  B2
```
And the command `jj rebase -s B -d B2`

We were previously marking B as abandoned, despite the comment stating that we were marking it as being succeeded by B2. This resulted in a call to `rewrite(rewrites={}, abandoned={B})` instead of `rewrite(rewrites={B=>B2}, abandoned={})`, which then made the new parent of `C` into `A` instead of `B2`
ilyagr added a commit to ilyagr/jj that referenced this issue Feb 3, 2024
This mostly reverts jj-vcs#2901 as well as its
fixup jj-vcs#2903. The related bug is reopened,
see jj-vcs#2869 (comment).

The problem is that while the fix did fix jj-vcs#2896 in most cases, it did
reintroduce the more severe bug jj-vcs#2760
in one case, if the working copy is the commit being rebased.

For example, suppose you have the tree

```
root -> A -> B -> @ (empty) -> C
```

### Before this commit

#### Case 1

`jj rebase -s B -d root` would work perfectly before this commit, resulting in

```
root -> A
  \-------B -> C
           \- @ (new, empty)
```

#### Case 2

Unfortunately, if you run `jj rebase -s @ -d A`, you'd have the following result
(before this commit), which shows the reintroduction of jj-vcs#2760:

```
root -> A @ -> C
         \-- B
```

with the working copy at `A`. The reason for this is explained in
jj-vcs#2901 (comment).

### After this commit

After this commit, both case 1 and case 2 will be wrong in the sense of jj-vcs#2896,
it will not exhibit the worse bug jj-vcs#2760.

Case 1 would result in:

```
root -> A
  \-------B -> @ (empty) -> C
```

Case 2 would result in:

```
root -> A -> @ -> C
         \-- B
```

with the working copy remaining a descendant of A
ilyagr added a commit to ilyagr/jj that referenced this issue Feb 3, 2024
This mostly reverts jj-vcs#2901 as well as its
fixup jj-vcs#2903. The related bug is reopened,
see jj-vcs#2869 (comment).

The problem is that while the fix did fix jj-vcs#2896 in most cases, it did
reintroduce the more severe bug jj-vcs#2760
in one case, if the working copy is the commit being rebased.

For example, suppose you have the tree

```
root -> A -> B -> @ (empty) -> C
```

### Before this commit

#### Case 1

`jj rebase -s B -d root --abandon-empty` would work perfectly before this
commit, resulting in

```
root -> A
  \-------B -> C
           \- @ (new, empty)
```

#### Case 2

Unfortunately, if you run `jj rebase -s @ -d A --abandon-empty`, you'd have the
following result (before this commit), which shows the reintroduction of jj-vcs#2760:

```
root -> A @ -> C
         \-- B
```

with the working copy at `A`. The reason for this is explained in
jj-vcs#2901 (comment).

### After this commit

After this commit, both case 1 and case 2 will be wrong in the sense of jj-vcs#2896,
but it will no longer exhibit the worse bug jj-vcs#2760 in the second case.

Case 1 would result in:

```
root -> A
  \-------B -> @ (empty) -> C
```

Case 2 would result in:

```
root -> A -> @ -> C
         \-- B
```

with the working copy remaining a descendant of A
ilyagr added a commit to ilyagr/jj that referenced this issue Feb 3, 2024
This mostly reverts jj-vcs#2901 as well as its
fixup jj-vcs#2903. The related bug is reopened,
see jj-vcs#2869 (comment).

The problem is that while the fix did fix jj-vcs#2896 in most cases, it did
reintroduce the more severe bug jj-vcs#2760
in one case, if the working copy is the commit being rebased.

For example, suppose you have the tree

```
root -> A -> B -> @ (empty) -> C
```

### Before this commit

#### Case 1

`jj rebase -s B -d root --abandon-empty` would work perfectly before this
commit, resulting in

```
root -> A
  \-------B -> C
           \- @ (new, empty)
```

#### Case 2

Unfortunately, if you run `jj rebase -s @ -d A --abandon-empty`, you'd have the
following result (before this commit), which shows the reintroduction of jj-vcs#2760:

```
root -> A @ -> C
         \-- B
```

with the working copy at `A`. The reason for this is explained in
jj-vcs#2901 (comment).

### After this commit

After this commit, both case 1 and case 2 will be wrong in the sense of jj-vcs#2896,
but it will no longer exhibit the worse bug jj-vcs#2760 in the second case.

Case 1 would result in:

```
root -> A
  \-------B -> @ (empty) -> C
```

Case 2 would result in:

```
root -> A -> @ -> C
         \-- B
```

with the working copy remaining a descendant of A
ilyagr added a commit to ilyagr/jj that referenced this issue Feb 3, 2024
This mostly reverts jj-vcs#2901 as well as its
fixup jj-vcs#2903. The related bug is reopened,
see jj-vcs#2869 (comment).

The problem is that while the fix did fix jj-vcs#2869 in most cases, it did
reintroduce the more severe bug jj-vcs#2760
in one case, if the working copy is the commit being rebased.

For example, suppose you have the tree

```
root -> A -> B -> @ (empty) -> C
```

### Before this commit

#### Case 1

`jj rebase -s B -d root --abandon-empty` would work perfectly before this
commit, resulting in

```
root -> A
  \-------B -> C
           \- @ (new, empty)
```

#### Case 2

Unfortunately, if you run `jj rebase -s @ -d A --abandon-empty`, you'd have the
following result (before this commit), which shows the reintroduction of jj-vcs#2760:

```
root -> A @ -> C
         \-- B
```

with the working copy at `A`. The reason for this is explained in
jj-vcs#2901 (comment).

### After this commit

After this commit, both case 1 and case 2 will be wrong in the sense of jj-vcs#2869,
but it will no longer exhibit the worse bug jj-vcs#2760 in the second case.

Case 1 would result in:

```
root -> A
  \-------B -> @ (empty) -> C
```

Case 2 would result in:

```
root -> A -> @ -> C
         \-- B
```

with the working copy remaining a descendant of A
ilyagr added a commit to ilyagr/jj that referenced this issue Feb 3, 2024
This mostly reverts jj-vcs#2901 as well as its
fixup jj-vcs#2903. The related bug is reopened,
see jj-vcs#2869 (comment).

The problem is that while the fix did fix jj-vcs#2869 in most cases, it did
reintroduce the more severe bug jj-vcs#2760
in one case, if the working copy is the commit being rebased.

For example, suppose you have the tree

```
root -> A -> B -> @ (empty) -> C
```

### Before this commit

#### Case 1

`jj rebase -s B -d root --skip-empty` would work perfectly before this
commit, resulting in

```
root -> A
  \-------B -> C
           \- @ (new, empty)
```

#### Case 2

Unfortunately, if you run `jj rebase -s @ -d A --skip-empty`, you'd have the
following result (before this commit), which shows the reintroduction of jj-vcs#2760:

```
root -> A @ -> C
         \-- B
```

with the working copy at `A`. The reason for this is explained in
jj-vcs#2901 (comment).

### After this commit

After this commit, both case 1 and case 2 will be wrong in the sense of jj-vcs#2869,
but it will no longer exhibit the worse bug jj-vcs#2760 in the second case.

Case 1 would result in:

```
root -> A
  \-------B -> @ (empty) -> C
```

Case 2 would result in:

```
root -> A -> @ -> C
         \-- B
```

with the working copy remaining a descendant of A
ilyagr added a commit that referenced this issue Feb 3, 2024
This mostly reverts #2901 as well as its
fixup #2903. The related bug is reopened,
see #2869 (comment).

The problem is that while the fix did fix #2869 in most cases, it did
reintroduce the more severe bug #2760
in one case, if the working copy is the commit being rebased.

For example, suppose you have the tree

```
root -> A -> B -> @ (empty) -> C
```

### Before this commit

#### Case 1

`jj rebase -s B -d root --skip-empty` would work perfectly before this
commit, resulting in

```
root -> A
  \-------B -> C
           \- @ (new, empty)
```

#### Case 2

Unfortunately, if you run `jj rebase -s @ -d A --skip-empty`, you'd have the
following result (before this commit), which shows the reintroduction of #2760:

```
root -> A @ -> C
         \-- B
```

with the working copy at `A`. The reason for this is explained in
#2901 (comment).

### After this commit

After this commit, both case 1 and case 2 will be wrong in the sense of #2869,
but it will no longer exhibit the worse bug #2760 in the second case.

Case 1 would result in:

```
root -> A
  \-------B -> @ (empty) -> C
```

Case 2 would result in:

```
root -> A -> @ -> C
         \-- B
```

with the working copy remaining a descendant of A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants