Checking existing uasset data during setting version after the load #180

moonyuet · 2024-12-16T14:49:16Z

Changelog Description

This PR is to add the missing check on existing uasset data during up-versioning or setting version after loading.
And this also allows the container data stores outside content directories.
Resolve #42

Additional clip to showcase how the enhancement looks like:
https://github.com/user-attachments/assets/c59e580d-7379-40b9-ac6e-094481ba693b

Additional review information

You can find more description of the issue in #42 (comment)

Testing notes:

Load any assets in Unreal
Move to different content path/plugins
Trying to set version/update to the latest

…versioning or setting version

LiborBatek · 2024-12-17T09:48:30Z

@moonyuet could u elaborate more on how to migrate assets and use as content plugin? Im not sure where shoud I migrate it and how....besides the obvious UE Migrate asset but lots of unclear things for me...
Im kinda stuck atm so any additional info might help a lot! thx

moonyuet · 2024-12-17T09:54:41Z

@moonyuet could u elaborate more on how to migrate assets and use as content plugin? Im not sure where shoud I migrate it and how....besides the obvious UE Migrate asset but lots of unclear things for me... Im kinda stuck atm so any additional info might help a lot! thx

You can test with the scenario like below:

Try to load the asset with multiple version (e.g. v48, v49, v50)
Move the objects to different folders
Set version(if you are at v50, try to set v48 as version) in Scene Inventory
Go back to the version folder of which related to the version you set to
You should find the assets (v48 asset and containers) into the folder.
Before this PR, setting version and up-versioning won't check the existing uasset data of which they would load a new asset into the version folder.

LiborBatek · 2024-12-17T10:22:32Z

I have tested as you wrote and my findings so far are:

After moving some asset version to other place (folder) the Set Version action wont perform the switching and asset stays at current version - no error shown.
during that Set version action no new asset version will be imported if already been within project elsewhere (e.g. on that new location) which is good news
once moving asset versions back to its original location (next to any existing version) the Set Version action starts working and asset get replaced by selected version without any issues...

LiborBatek · 2024-12-17T10:27:37Z

Im enclosing the log when replacing the version with other (moved to other location) is not working but not failing with error:

LogPython: old_content
LogPython: []
LogPython: static_mesh_comps
LogPython: [<Object '/Game/test.test:PersistentLevel.StaticMeshActor_UAID_00155D9B44C5B33902_1265224191.StaticMeshComponent0' (0x00000680BFB51C00) Class 'StaticMeshComponent'>]
LogPython: old_meshes
LogPython: {}
LogPython: new_meshes
LogPython: {}
Message dialog closed, result: Ok, title: Unable to Check Out From Revision Control!, text: The following assets could not be successfully checked out from revision control:
/Game/test
Cmd: OBJ SAVEPACKAGE PACKAGE="/Game/test" FILE="../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap" SILENT=true AUTOSAVING=false KEEPDIRTY=false
LogUObjectHash: Compacting FUObjectHashTables data took   0.45ms
LogSavePackage: Moving output files for package: /Game/test
LogSavePackage: Moving '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Saved/testB93C42734E522CEED7A403AC8AF60785.tmp' to '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap'
LogFileHelpers: All files are already saved.
LogPython: old_content
LogPython: ["/Game/Ayon/assets/props/container/modelMain_v003_abc/container_modelMain_v003_abc.container_modelMain_v003_abc", "/Game/Ayon/assets/props/container/modelMain_v003_abc/modelMain_v003_abc_CON.modelMain_v003_abc_CON"]
LogPython: static_mesh_comps
LogPython: [<Object '/Game/test.test:PersistentLevel.StaticMeshActor_UAID_00155D9B44C5B33902_1265224191.StaticMeshComponent0' (0x00000680BFB51C00) Class 'StaticMeshComponent'>]
LogPython: old_meshes
LogPython: {'container_modelMain_v__abc': <Object '/Game/Ayon/assets/props/container/modelMain_v003_abc/container_modelMain_v003_abc.container_modelMain_v003_abc' (0x00000680DC3DA800) Class 'StaticMesh'>}
LogPython: new_meshes
LogPython: {'container_modelMain_v__abc': <Object '/Game/Ayon/assets/props/container/modelMain_v003_abc/container_modelMain_v003_abc.container_modelMain_v003_abc' (0x00000680DC3DA800) Class 'StaticMesh'>}
Message dialog closed, result: Ok, title: Unable to Check Out From Revision Control!, text: The following assets could not be successfully checked out from revision control:
/Game/test
Cmd: OBJ SAVEPACKAGE PACKAGE="/Game/test" FILE="../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap" SILENT=true AUTOSAVING=false KEEPDIRTY=false
LogUObjectHash: Compacting FUObjectHashTables data took   0.24ms
LogSavePackage: Moving output files for package: /Game/test
LogSavePackage: Moving '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Saved/test930E7F1546805DCFC4276A9488A3F512.tmp' to '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap'
LogFileHelpers: All files are already saved.
LogContentValidation: Display: Starting to validate 1 assets
LogContentValidation: Enabled validators:
LogContentValidation:     /Script/DataValidation.EditorValidator_Material
LogContentValidation:     /Script/DataValidation.DirtyFilesChangelistValidator
LogContentValidation:     /Script/DataValidation.EditorValidator_Localization
LogContentValidation:     /Script/DataValidation.PackageFileValidator
LogContentValidation:     /Script/DataValidation.WorldPartitionChangelistValidator
LogContentValidation:     /Script/InputBlueprintNodes.EnhancedInputUserWidgetValidator
AssetCheck: /Game/test Validating asset

moonyuet · 2024-12-18T07:07:57Z

I have tested as you wrote and my findings so far are:

After moving some asset version to other place (folder) the Set Version action wont perform the switching and asset stays at current version - no error shown.

during that Set version action no new asset version will be imported if already been within project elsewhere (e.g. on that new location) which is good news

once moving asset versions back to its original location (next to any existing version) the Set Version action starts working and asset get replaced by selected version without any issues...

Try not to move the container data (the red squared) along with the asset.
good
See 1.

moonyuet · 2024-12-18T07:08:30Z

Im enclosing the log when replacing the version with other (moved to other location) is not working but not failing with error:

LogPython: old_content
LogPython: []
LogPython: static_mesh_comps
LogPython: [<Object '/Game/test.test:PersistentLevel.StaticMeshActor_UAID_00155D9B44C5B33902_1265224191.StaticMeshComponent0' (0x00000680BFB51C00) Class 'StaticMeshComponent'>]
LogPython: old_meshes
LogPython: {}
LogPython: new_meshes
LogPython: {}
Message dialog closed, result: Ok, title: Unable to Check Out From Revision Control!, text: The following assets could not be successfully checked out from revision control:
/Game/test
Cmd: OBJ SAVEPACKAGE PACKAGE="/Game/test" FILE="../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap" SILENT=true AUTOSAVING=false KEEPDIRTY=false
LogUObjectHash: Compacting FUObjectHashTables data took   0.45ms
LogSavePackage: Moving output files for package: /Game/test
LogSavePackage: Moving '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Saved/testB93C42734E522CEED7A403AC8AF60785.tmp' to '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap'
LogFileHelpers: All files are already saved.
LogPython: old_content
LogPython: ["/Game/Ayon/assets/props/container/modelMain_v003_abc/container_modelMain_v003_abc.container_modelMain_v003_abc", "/Game/Ayon/assets/props/container/modelMain_v003_abc/modelMain_v003_abc_CON.modelMain_v003_abc_CON"]
LogPython: static_mesh_comps
LogPython: [<Object '/Game/test.test:PersistentLevel.StaticMeshActor_UAID_00155D9B44C5B33902_1265224191.StaticMeshComponent0' (0x00000680BFB51C00) Class 'StaticMeshComponent'>]
LogPython: old_meshes
LogPython: {'container_modelMain_v__abc': <Object '/Game/Ayon/assets/props/container/modelMain_v003_abc/container_modelMain_v003_abc.container_modelMain_v003_abc' (0x00000680DC3DA800) Class 'StaticMesh'>}
LogPython: new_meshes
LogPython: {'container_modelMain_v__abc': <Object '/Game/Ayon/assets/props/container/modelMain_v003_abc/container_modelMain_v003_abc.container_modelMain_v003_abc' (0x00000680DC3DA800) Class 'StaticMesh'>}
Message dialog closed, result: Ok, title: Unable to Check Out From Revision Control!, text: The following assets could not be successfully checked out from revision control:
/Game/test
Cmd: OBJ SAVEPACKAGE PACKAGE="/Game/test" FILE="../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap" SILENT=true AUTOSAVING=false KEEPDIRTY=false
LogUObjectHash: Compacting FUObjectHashTables data took   0.24ms
LogSavePackage: Moving output files for package: /Game/test
LogSavePackage: Moving '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Saved/test930E7F1546805DCFC4276A9488A3F512.tmp' to '../../../../../../projects/Quantum_Demo/unreal/Generic/QD_UE/Content/test.umap'
LogFileHelpers: All files are already saved.
LogContentValidation: Display: Starting to validate 1 assets
LogContentValidation: Enabled validators:
LogContentValidation:     /Script/DataValidation.EditorValidator_Material
LogContentValidation:     /Script/DataValidation.DirtyFilesChangelistValidator
LogContentValidation:     /Script/DataValidation.EditorValidator_Localization
LogContentValidation:     /Script/DataValidation.PackageFileValidator
LogContentValidation:     /Script/DataValidation.WorldPartitionChangelistValidator
LogContentValidation:     /Script/InputBlueprintNodes.EnhancedInputUserWidgetValidator
AssetCheck: /Game/test Validating asset

I will fix it in the upcoming commit

LiborBatek · 2024-12-18T10:43:51Z

still even if keeping the _CON in versions folder doesnt help and Set version wont happen properly...

Still behave the same as when moving the full version folder to different location...

LiborBatek · 2024-12-18T10:46:01Z

I really dont know whats the intention of this one....as Migrate action pushes to different project and just moving asset within active project doesnt make much sense in practical use.

But still malfunctioning on Manage>Replace Actor with selected

moonyuet · 2024-12-18T14:16:05Z

still even if keeping the _CON in versions folder doesnt help and Set version wont happen properly...

Still behave the same as when moving the full version folder to different location...

Maybe I dont explain well about the migration tool. This video might help a little bit. The fix is not really related to how the migration tool works with the asset (It's actually working pretty well within the task project after the enhancement on the scene inventory, I didn't try to migrate this asset into different project) but how the up-versioning and setting version can check on the existing uasset data. (It's not working for the switching asset)
https://github.com/user-attachments/assets/d0a98491-89ce-4a6e-a01e-3e786ff7ff80

But still malfunctioning on Manage>Replace Actor with selected

We can probably introduce some more info to make to check works also with the related existing uasset data with the inventory tool. If you want to
I also find out that we are still using old getting representation path, so this could be the reason you can't load the container in other projects

antirotor

So I've tested it in the Content Plugin scenario. TL;DR: it doesn't work as it should. This is the result:

The v003 is newly loaded model, v001 is the model migrated to content plugin. What I would expect is that new version will be loaded to the content plugin (or the existing one should be updated, if not used elsewhere).

More info about how to replicate

create the Content Plugin - In Unreal, open plugins manager. Top right there is a green button "+add", select Content Plugin, fill in the required stuff, select the destination folder for the plugin (outside the project).
loade something to Unreal - load something from AYON to Unreal
migrate the loaded asset to Content plugin - select the folder you've just loaded in content plugin, right click, hit migrate, select the folder in your new content plugin and finish migration. Delete the original asset in the project if it is still there. At this point, the Scene Manager should still see the loaded asset.
Publish new version of that stuff and try to load it, or switch the version on it. New data should end up in the content plugin and not in the project.

LiborBatek

sry wrong post...had been removed

posted to wrong PR

moonyuet · 2024-12-23T13:20:00Z

@antirotor I have implemented the changes, and the asset would be loaded to the content plugins instead if there is one.

LiborBatek

I have been through all the steps 1 - 4 succesful and as seen on the scrngrab I could still see the loaded asset which was migrated via content plugin to external folder 👍

Scene Inventory lists the loaded asset even tho content folder been empty (as these are comming from the content plugin)

When inspecting the content folder (content plugins one) there is probably an issue once triggering Manage > Set Version as the newly loaded asset data sits next to the original version (not sure how this should be working- probably replacing the orig files?)

as you can see v001 and v002 been in different folder

and also when triggering Ayon > Load for bringing fresh new asset version... again inspecting the content folder the newly loaded version (in my case v003 ) sits off the folder for the prop completely - which is odd too.

should be inside root asset folder at least (marked by green arrow)

PS. : pls dont be confused by the asset name consisting of container_scifi_02 as its asset name not version, rather pointing it out to not confuse you - yeah I should have picked different asset before testing sry :)

…he content plugin

supports to check on the existing uasset when loading data during up-…

5bf4a57

…versioning or setting version

moonyuet requested review from antirotor and LiborBatek December 16, 2024 14:49

moonyuet self-assigned this Dec 16, 2024

moonyuet added the type: enhancement Improvement of existing functionality or minor addition label Dec 16, 2024

moonyuet added 3 commits December 18, 2024 22:35

up-versioning work across different projects

f78de09

ensure the asset is being imported/hooked back to version folder

e7e98be

remove unused variable

ff86be4

antirotor requested changes Dec 19, 2024

View reviewed changes

moonyuet added 2 commits December 19, 2024 21:40

include the changes on the Plugins Folders

97859c4

restore some codes for pipeline.py

f9d67b2

moonyuet marked this pull request as draft December 20, 2024 10:16

moonyuet added 4 commits December 20, 2024 22:17

add functions for adding the asset to the content plugin

3ee6558

remove unused import

7ec3e76

add the migration of content plugin functionn into the asset loaders

fa6b030

use content plugin path as loading asset directories if there is one

46180aa

moonyuet marked this pull request as ready for review December 23, 2024 10:37

moonyuet requested a review from antirotor December 23, 2024 10:37

LiborBatek previously requested changes Dec 23, 2024

View reviewed changes

LiborBatek self-requested a review December 23, 2024 11:34

LiborBatek requested changes Dec 30, 2024

View reviewed changes

moonyuet added 2 commits December 30, 2024 22:41

make sure the content loaded in the correct content plugin

0f941dc

make sure the uasset and yeti loading into the correct directory of t…

6a1abac

…he content plugin

moonyuet requested a review from LiborBatek December 30, 2024 15:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Checking existing uasset data during setting version after the load #180

Checking existing uasset data during setting version after the load #180

moonyuet commented Dec 16, 2024 •

edited

Loading

LiborBatek commented Dec 17, 2024

moonyuet commented Dec 17, 2024

LiborBatek commented Dec 17, 2024

LiborBatek commented Dec 17, 2024

moonyuet commented Dec 18, 2024

moonyuet commented Dec 18, 2024

LiborBatek commented Dec 18, 2024

LiborBatek commented Dec 18, 2024

moonyuet commented Dec 18, 2024 •

edited

Loading

antirotor left a comment •

edited

Loading

LiborBatek left a comment •

edited

Loading

moonyuet commented Dec 23, 2024

LiborBatek left a comment •

edited

Loading

Checking existing uasset data during setting version after the load #180

Are you sure you want to change the base?

Checking existing uasset data during setting version after the load #180

Conversation

moonyuet commented Dec 16, 2024 • edited Loading

Changelog Description

Additional review information

Testing notes:

LiborBatek commented Dec 17, 2024

moonyuet commented Dec 17, 2024

LiborBatek commented Dec 17, 2024

LiborBatek commented Dec 17, 2024

moonyuet commented Dec 18, 2024

moonyuet commented Dec 18, 2024

LiborBatek commented Dec 18, 2024

LiborBatek commented Dec 18, 2024

moonyuet commented Dec 18, 2024 • edited Loading

antirotor left a comment • edited Loading

Choose a reason for hiding this comment

LiborBatek left a comment • edited Loading

Choose a reason for hiding this comment

moonyuet commented Dec 23, 2024

LiborBatek left a comment • edited Loading

Choose a reason for hiding this comment

moonyuet commented Dec 16, 2024 •

edited

Loading

moonyuet commented Dec 18, 2024 •

edited

Loading

antirotor left a comment •

edited

Loading

LiborBatek left a comment •

edited

Loading

LiborBatek left a comment •

edited

Loading