XWIKI-12987: Relative links are made absolute or even broken after moving a page #3553

surli · 2024-10-10T15:54:20Z

Jira URL

https://jira.xwiki.org/browse/XWIKI-12987

TODO

Document properly abstract methods in AbstractCopyOrMoveJob
Improve SubWikiIT#movePageToSubwiki
Why not checking if the doc exists in updateResourceReferenceAbsolute conditions?

Changes

Description

The idea of this work is to:

Provide a way to access all documents that are moved as part of a move job
Use that information when performing a call to ReferenceRenamer to define if a relative untyped link should be handled or not

The PR provides mainly:

new APIs in ReferenceRenamer and MacroRefactoring to integrate the set of references that have been moved as part of same job
new API in AbstractEntityJobWithChecks to be able to retrieve the list of references that will be effectively refactored (the ones selected in case of questions)
small refactoring of AbstractCopyOrMoveJob with new abstract methods to avoid duplications (not strictly needed for this work)
new conditions in ResourceReferenceRenamer to decide if a link should be renamed or not: most of the logic of the fix is encoded there (see also clarifications)
new calls in XWiki#updateLinksForRename and BackLinkUpdaterListener#updateBackLinks to give the set of documents when calling the rename of references
new integration test simulating the scenario indicated in the ticket and also performing a supplementary check related to a regression found afterwards

Clarifications

The refactoring of references is currently called at two places:

by the BackLinkUpdaterListener for all backlinks after a document has been renamed (triggered by a document event)
by XWiki#updateLinksForRename to rename the internal links of the current document (which is always a call to updateResourceReferenceRelative, see below)

The problem of XWIKI-12987 is that XWiki#updateLinksForRename is called first and does perform an absolute rename of the relative links.
Now ResourceReferenceRenamer APIs names might be misleading: updateResourceReferenceRelative and updateResourceReferenceAbsolute are not about the references being absolute or relative: it's about the renamed references being absolute or relative respectively to the current document. It took me a while to integrate this, and I'm still struggling a bit with it.

So the problem was to find a proper condition to decide when to not refactor links, for this I'm performing a check for assessing if a link is absolute or not, by trying to resolve the ResourceReference without any parameter: if the result equals the reference with parameter then it was absolute.

Then for the updateResourceReferenceAbsolute the idea is to only perform update of the links, if the provided link is absolute, or if it's relative but the current document hasn't been moved as part of same job: in such case we do need to update the relative link, because there won't be a call to XWiki#updateLinksForRename on that document to update the link, we only get the call from BackLinkUpdaterListener.

For the updateResourceReferenceRelative the check is a bit more complex.
We only update links that are relative here, we don't want to update absolute references (is that correct? Can't find a counter example right now).
Then since we only perform refactoring of links relative to current document, we also check that the link about to be refactored is not related to pages that are part of the moved document in the same job: if those are also moved in the same job, then they're moved using same "direction", they're part of same hierarchy and we don't want to change the relative links wrt to them. This check is the main part of avoiding to update the relative links.

And finally we perform the update of the link only if the doc actually exists: we would create absolute links for those not existing doc, which doesn't make sense, we should keep the relative link we don't really know what the user wanted to do with those. Note that we could do the same check in updateResourceReferenceAbsolute but we don't really have the need since this is only called from the BackLinkUpdaterListener and if I'm correct we'll never have registered backlinks for a not existing doc.

Note that initially we discussed about using untyped link as a condition to perform or not the refactoring: I dropped the idea because we currently always create image resource references as untyped references from the WYSIWYG editor.

Screenshots & Video

Executed Tests

Run of tests on following modules / integration test:

xwiki-platform-refactoring
xwiki-platform-flamingo-skin-test-docker
xwiki-platform-attachment-test-docker and specifically AttachmentMoveIT
xwiki-platform-index-test-docker and specifically AllDocsIT
xwiki-platform-wiki-test-docker and specifically SubWikiIT

Expected merging strategy

Prefers squash: Yes
Backport on branches:
*

tmortagne · 2024-10-30T14:25:00Z

...tform-flamingo-skin-test-docker/src/test/it/org/xwiki/flamingo/test/docker/RenamePageIT.java

-            setup.createPage(sourcePageReference2, "Some content to be linked in macro. number 2");
-        ViewPage nestedMacroLinkPage =
-            setup.createPage(sourcePageReference3, "Some content to be linked in nested macro. number 3");
+        setup.createPage(sourcePageReference1, "Some content to be linked. number 1");


Better use the rest API if you don't need a ViewPage, it's much faster and reliable.

tmortagne · 2024-10-30T14:28:07Z

...tform-flamingo-skin-test-docker/src/test/it/org/xwiki/flamingo/test/docker/RenamePageIT.java

+
+        // Wait for an empty queue here to ensure that the deleted page has been removed from the index and links
+        // won't be updated just because the page is still in the index.
+        new SolrTestUtils(testUtils, testConfiguration.getServletEngine()).waitEmptyQueue();


This should not be needed anymore with XWIKI-22323.

I actually call it to avoid getting a question when performing the rename operation.

The question is supposed to automatically vanish (it's not a blocker question) as soon as the job is done AFAIU.

well what's sure is that I'm getting a timeout when I remove this and the screenshot shows the question...

We might need to update the PO to support non-blocking questions (maybe it does not find what it expects to see to check if the job is still running when a question is displayed).

tmortagne · 2024-10-30T14:33:51Z

...ing/xwiki-platform-refactoring-api/src/main/java/org/xwiki/refactoring/ReferenceRenamer.java

+     * @param oldTarget the previous reference of the renamed entity (attachment or document)
+     * @param newTarget the new reference of the renamed entity (attachment or document)
+     * @param relative {@code true} if the link should be serialized relatively to the current document
+     * @param updatedDocuments the list of documents that have been renamed in the same job: this list contains the


I'm not sure about the "have been", isn't this called during the rename, for each page ? Also, I think it's related to copy too and not just rename, right ?

tmortagne · 2024-10-30T14:37:52Z

...ing/xwiki-platform-refactoring-api/src/main/java/org/xwiki/refactoring/ReferenceRenamer.java

+    @Unstable
+    default boolean renameReferences(Block block, DocumentReference currentDocumentReference,
+        DocumentReference oldTarget,
+        DocumentReference newTarget, boolean relative, Set<DocumentReference> updatedDocuments)


updatedDocuments does not seem enough to me. I don't see how you can properly cover all use cases without knowing where those pages are moved.

tmortagne · 2024-10-30T14:39:12Z

...-refactoring-api/src/main/java/org/xwiki/refactoring/internal/job/AbstractCopyOrMoveJob.java

+            if (((CancelableEvent) beginEvent).isCanceled()) {
+                return;
+            }
+            this.progressManager.endStep(this);


You don't need to call endStep if you have a startStep with the same source.

tmortagne · 2024-10-30T14:39:56Z

...-refactoring-api/src/main/java/org/xwiki/refactoring/internal/job/AbstractCopyOrMoveJob.java

+            this.progressManager.startStep(this);
+            EndFoldEvent endEvent = getEndEvent();
+            this.observationManager.notify(endEvent, this, this.getRequest());
+            this.progressManager.endStep(this);


You don't need this endStep, it will be closed by popLevelProgress.

I don't need it but it's not a mistake to have it right? Note that here it's some old code that I just moved, and personally I feel more comfortable to keep the symmetry start/end Step

It's not going to cause problems, it's just useless.

I feel more comfortable to keep the symmetry start/end Step

On my side, I usually prefer to limit plumbing to not pollute too much the "real" code.

tmortagne · 2024-10-30T14:44:23Z

...xwiki-platform-refactoring-api/src/main/java/org/xwiki/refactoring/internal/job/CopyJob.java

@@ -90,4 +66,16 @@ protected boolean atomicOperation(DocumentReference source, DocumentReference ta
    {
        return this.modelBridge.copy(source, target);
    }
+
+    @Override
+    protected <T extends BeginFoldEvent & CancelableEvent> T getBeginEvent()


Feels more like a createBeginEvent.

tmortagne · 2024-10-30T14:44:33Z

...xwiki-platform-refactoring-api/src/main/java/org/xwiki/refactoring/internal/job/CopyJob.java

+    }
+
+    @Override
+    protected EndFoldEvent getEndEvent()


Same comment as above.

tmortagne · 2024-10-30T14:48:15Z

...xwiki-platform-refactoring-api/src/main/java/org/xwiki/refactoring/internal/job/CopyJob.java

@@ -90,4 +66,16 @@ protected boolean atomicOperation(DocumentReference source, DocumentReference ta
    {
        return this.modelBridge.copy(source, target);
    }
+
+    @Override
+    protected <T extends BeginFoldEvent & CancelableEvent> T getBeginEvent()


Since the type of begin event and end event depend on the class implementing and not on the code calling the method, you should declare those two type in the class definition.

Alternatively, you could just declare that getBeginEvent returns a BeginFoldEvent and check in the abstract if it happens to also be a CancelableEvent. Would be more consistent with getEndEvent.

…ving a page WIP The idea of this work is to: 1. Provide a way to access all documents that are moved as part of a move job 2. Use that information when performing a call to ReferenceRenamer to define if a relative untyped link should be handled or not On top of it, the idea is also to check if the doc exists in case of refactoring of a link to avoid refactoring unexisting relative links. One problem is remaining about relative link pointing to sibling pages (e.g. the link to Alice in Bob page in the ticket): we rely apparently to an old mechanism for backward compatibility reason for this to work in the UI, we might need same thing in the check, or to decide to ignore that UC. I started to add an integration tests but for some reason it's not passing, though it seemed to be working locally for the scenario described in the ticket (except for the link in Bob page).

…ving a page * Fix integration test setup * Fix some signatures * Work on the conditions for performing link update: WIP

…ving a page * Fix conditions to make all RenamePageIT passing * WIP: need to double check that some conditions are not redundant and double check side effects

…ving a page * Simplify a bit the conditions in ResourceReferenceRenamer and ensure all unit tests are passing in refactoring module

…ving a page * Fix checkstyle * WIP: try to find proper oracle for renaming absolute references, without success so far.

…ving a page * Find proper conditions to perform or not link renames * Fix unit tests to add missing conditions * WIP: need to fix coverage and check on subwikis / with more conditions (e.g. with holes in hierarchy)

…ving a page * Fix a regression and provide a test to cover it

…ving a page * Provide subwiki integration tests * Minor improvment in RenamePageIT

…ving a page * Improve SubWikiIT to add more checks

surli self-assigned this Oct 10, 2024

surli force-pushed the XWIKI-12987 branch 2 times, most recently from bd6d6fe to 12a6e52 Compare October 23, 2024 16:24

surli marked this pull request as ready for review October 29, 2024 14:32

surli force-pushed the XWIKI-12987 branch from 1e5ef33 to 4b1f9bf Compare October 29, 2024 16:55

surli requested a review from tmortagne October 30, 2024 08:49

tmortagne reviewed Oct 30, 2024

View reviewed changes

surli added 8 commits October 31, 2024 12:06

XWIKI-12987: Relative links are made absolute or even broken after mo…

d9d5d0f

…ving a page * Fix integration test setup * Fix some signatures * Work on the conditions for performing link update: WIP

XWIKI-12987: Relative links are made absolute or even broken after mo…

4b6e0ab

…ving a page * Fix conditions to make all RenamePageIT passing * WIP: need to double check that some conditions are not redundant and double check side effects

XWIKI-12987: Relative links are made absolute or even broken after mo…

f74c4c5

…ving a page * Simplify a bit the conditions in ResourceReferenceRenamer and ensure all unit tests are passing in refactoring module

XWIKI-12987: Relative links are made absolute or even broken after mo…

3f16fb6

…ving a page * Fix checkstyle * WIP: try to find proper oracle for renaming absolute references, without success so far.

XWIKI-12987: Relative links are made absolute or even broken after mo…

c5904bc

…ving a page * Fix a regression and provide a test to cover it

XWIKI-12987: Relative links are made absolute or even broken after mo…

c58c5e6

…ving a page * Provide subwiki integration tests * Minor improvment in RenamePageIT

surli force-pushed the XWIKI-12987 branch from 4b1f9bf to c58c5e6 Compare October 31, 2024 11:28

XWIKI-12987: Relative links are made absolute or even broken after mo…

ff94733

…ving a page * Improve SubWikiIT to add more checks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XWIKI-12987: Relative links are made absolute or even broken after moving a page #3553

XWIKI-12987: Relative links are made absolute or even broken after moving a page #3553

surli commented Oct 10, 2024 •

edited

Loading

tmortagne Oct 30, 2024 •

edited

Loading

tmortagne Oct 30, 2024

surli Oct 31, 2024

tmortagne Oct 31, 2024 •

edited

Loading

surli Oct 31, 2024

tmortagne Oct 31, 2024 •

edited

Loading

tmortagne Oct 30, 2024 •

edited

Loading

tmortagne Oct 30, 2024

tmortagne Oct 30, 2024

tmortagne Oct 30, 2024

surli Oct 31, 2024

tmortagne Nov 2, 2024

tmortagne Oct 30, 2024

tmortagne Oct 30, 2024

tmortagne Oct 30, 2024 •

edited

Loading

XWIKI-12987: Relative links are made absolute or even broken after moving a page #3553

Are you sure you want to change the base?

XWIKI-12987: Relative links are made absolute or even broken after moving a page #3553

Conversation

surli commented Oct 10, 2024 • edited Loading

Jira URL

TODO

Changes

Description

Clarifications

Screenshots & Video

Executed Tests

Expected merging strategy

tmortagne Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tmortagne Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tmortagne Oct 31, 2024 • edited Loading

Choose a reason for hiding this comment

tmortagne Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tmortagne Oct 30, 2024 • edited Loading

Choose a reason for hiding this comment

surli commented Oct 10, 2024 •

edited

Loading

tmortagne Oct 30, 2024 •

edited

Loading

tmortagne Oct 31, 2024 •

edited

Loading

tmortagne Oct 31, 2024 •

edited

Loading

tmortagne Oct 30, 2024 •

edited

Loading

tmortagne Oct 30, 2024 •

edited

Loading