Bug: almost every edit results in divergence

[avamsi]

Description

• This is a collocated repo.
• I have a jj log running in the background every couple seconds.

Steps to Reproduce the Problem

I'm unable to reproduce the issue in an almost empty test repo.

[avamsi]

It isn't a particularly large repo as well --

$ jj debug index

Number of commits: 167023
Number of merges: 32
Max generation number: 130228
Number of heads: 3621
Number of changes: 166995

(PLMK if there's any debug info I can provide.)

[yuja]

I think this is similar to #924. The background jj log would capture the intermediate state. jj op log will help to debug which operation created divergence.

If you're okay with jj log --ignore-working-copy, you can avoid various concurrency problems.

[avamsi]

print "@ Working copy changes:"
jj --color=always diff --revision=@ --summary | indent
print "\n| Parent commit changes:"
jj --color=always --ignore-working-copy diff --revision=@- --summary | indent
print
jj --color=always --ignore-working-copy log

This is what I actually run every couple seconds -- the last two jj invocations are with --ignore-working-copy but the first one is not because I actually want it to snapshot the working copy so I can see the changes.

Re: "background jj log would capture the intermediate state" (and #924 (comment)), doesn't jj have locking to prevent this? (Just to be clear, I don't mean to say you're wrong -- just trying to understand the bug.)

[yuja]

doesn't jj have locking to prevent this?

jj has locks (most notably for working copy updates), but tries to minimize the locked section afaik. Concurrent operations will be merged later and eventually converge.

https://github.com/martinvonz/jj/blob/main/docs/technical/concurrency.md

I suspect that modification on git storage would be visible a bit earlier, and captured by automated git import on colocated repo.

[martinvonz]

@avamsi, does jj debug reindex help restore your repo to a working state? I hope this is not blocking you

[avamsi]

jj debug reindex doesn't seem to help. And yeah, it's not blocking any of my workflows, it's just slightly annoying.

To give a little bit more context, I don't actually do any concurrent editing, so unless I checkout a different revision accidentally, my working copy would be the actual state, so I can just hide all other divergent revisions by running something like jj hide $(jj commits -r "$(jj changes -r @) ~ @") -- I'm using some aliases but hopefully the intent is clear.

[avamsi]

This was a pleasant surprise! Thanks, Yuya! One early observation, -- looks like I'm ending up with 2 empty revisions now (which is very much a non issue, especially in comparison) in cases where I would end up with 2 divergent revisions before. I may be wrong though, I'll update again in some time.

[yuja]

looks like I'm ending up with 2 empty revisions now

I observed such (random) behavior while debugging #924, but my test script spawned ~40 jj status & processes. I couldn't reliably reproduce the problem with one jj status & + jj squash &.

A possible explanation is that the jj status process captured the HEAD ref just moved by e.g. jj squash, and created new working-copy commit. Since jj squash should create its own working-copy commit, the end result is two empty head commits. However, snapshot() runs import_git_refs_and_head() first followed by snapshot_working_copy(). Suppose snapshot_working_copy() is the synchronization point of two processes (it acquires lock, and snapshot would take some time), I don't think this scenario occurs frequently.

https://github.com/martinvonz/jj/blob/c23f1e4161eac10b49d77a0c18e2d9e6649e20c2/src/cli_util.rs#L652-L660

[avamsi]

Sorry, I forgot to update this.

I don't think this scenario occurs frequently.

Yep, can confirm -- didn't face this again after my previous comment!