Maya: Refactor moved usage of CreateRender settings

[BigRoy]

Brief description

This is a draft untested PR to fix #3642

Description

Please test publishing from Maya with up-to-date saved settings.

Testing

1. Ensure render images file rule works as expected.
2. Ensure aov separator works as expected

[BigRoy]

Both before and after the code change I feel the except KeyError code isn't the most "explicit" we have now. We are currently unable to identify whether the key error is due to the self._aov_chars dictionary not having the right key or whether the Project Settings didn't have the stored setting key.

I feel like both are different errors and potentially should be handled differently. No?

[BigRoy]

Works for us now.

Only thing still missing to be able to submit renders from Maya again is the actual populating of full_exp_files and thus expectedFiles remaining empty currently. It's missing this line that was there originally.

This commit would fix it.

Wasn't sure whether it made sense to include it in this PR.

[m-u-r-p-h-y]

shouldn't we put some default value here?

[m-u-r-p-h-y]

still not passing through

// Error: pyblish.plugin : Traceback (most recent call last):
  File "D:\REPO\OpenPype\.venv\lib\site-packages\pyblish\plugin.py", line 522, in __explicit_process
    runner(*args)
  File "D:\REPO\OpenPype\openpype\hosts\maya\plugins\publish\validate_render_image_rule.py", line 30, in process
AssertionError: Workspace's `images` file rule must be set to: renders/
Traceback (most recent call last):
  File "D:\REPO\OpenPype\.venv\lib\site-packages\pyblish\plugin.py", line 522, in __explicit_process
    runner(*args)
  File "<string>", line 30, in process
AssertionError: Workspace's `images` file rule must be set to: renders/ // 

[BigRoy]

still not passing through

I believe you should set it to just renders without the slash? (We could strip the slash in the validator if you'd want to support ending with a slash in settings as well.)

shouldn't we put some default value here?

I'd just put whatever the default workspace.mel sets that comes with OP, which is indeed renders

[m-u-r-p-h-y]

Yes, it is the slash so we should change the wording in the validator message

[BigRoy]

Yes, it is the slash so we should change the wording in the validator message

Technically this would also work renders/lighting or whatever subfolders you'd like.
What wording would describe it better for you?

Might just be best to strip any starting/trailing slashes when checking against it to avoid user error anyway?

Also I'm not sure whether the workspace.mel actually allows storing the value with a slash at the end - if it does then technically it could also explicitly check the value directly with the trailing slash.

[m-u-r-p-h-y]

as a "user" I was strictly following the validators messages to use the trailing slash:

Validates "images" file rule is set to "renders/"
and
Workspace's images file rule must be set to: renders/

[BigRoy]

@m-u-r-p-h-y How about like this? (Just configured it to foobar to test)

[BigRoy]

Changed default value and changed the validator's wording.

Might just be best to strip any starting/trailing slashes when checking against it to avoid user error anyway?

Would you still like me to update the validator for this?

[BigRoy]

Another question. Is there any reason why this:

def get_file_rule(rule):
    """Workaround for a bug in python with cmds.workspace"""
    return mel.eval('workspace -query -fileRuleEntry "{}"'.format(rule))

value = get_file_rule("images")

Isn't just this:

value = cmds.workspace(fileRuleEntry="images")

---

And is there a reason why we're setting the workspace rule with Pymel using:

        pm.workspace.fileRules["images"] = default
        pm.system.Workspace.save()

Instead of doing this:

cmds.workspace(fileRule=("images", default))
cmds.workspace(saveWorkspace=True)

Especially since we might want to use as little Pymel as possible also with things like #2353 and #3233

---

Implemented these changes with 8fe2048 - please test.

[m-u-r-p-h-y]

Might just be best to strip any starting/trailing slashes when checking against it to avoid user error anyway?

Would you still like me to update the validator for this?

I'd just treat it the standard way we do it in other places where we deal with paths. No special treatment.

[m-u-r-p-h-y]

render images file rule works as expected now

I have some troubles with aov separator:
(trying to break it on purpose)

set the aov separator to - (dash)
created couple of aovs

this is the default Redshift aov settings:
<BeautyPath>/<BeautyFile>.<RenderPass>

but the error I got from Render Settings validator

// Error: pyblish.ValidateRenderSettings : AOV (BeautyAux) image prefix is not set correctly <BeautyPath>/<BeautyFile>.<RenderPass> != <BeautyPath>/<BeautyFile>_<RenderPass> // 
// Error: pyblish.ValidateRenderSettings : AOV (Cryptomatte) image prefix is not set correctly <BeautyPath>/<BeautyFile>.<RenderPass> != <BeautyPath>/<BeautyFile>_<RenderPass> // 
// Error: pyblish.ValidateRenderSettings : AOV file format is not the same as the one set globally 1 != 6 // 
// Error: pyblish.ValidateRenderSettings : AOV (Z) image prefix is not set correctly <BeautyPath>/<BeautyFile>.<RenderPass> != <BeautyPath>/<BeautyFile>_<RenderPass> // 
// Error: pyblish.plugin : Traceback (most recent call last):
  File "D:\REPO\OpenPype\.venv\lib\site-packages\pyblish\plugin.py", line 522, in __explicit_process
    runner(*args)
  File "D:\REPO\OpenPype\openpype\hosts\maya\plugins\publish\validate_rendersettings.py", line 91, in process
AssertionError: Invalid render settings found for 'Main'!
Traceback (most recent call last):
  File "D:\REPO\OpenPype\.venv\lib\site-packages\pyblish\plugin.py", line 522, in __explicit_process
    runner(*args)
  File "<string>", line 91, in process
AssertionError: Invalid render settings found for 'Main'! // 

is using underscore

also, the file format error is not very clear, as the user is not aware of 1 and 6 and what format it actually represents.

[BigRoy]

Nice! :)

---

// Error: pyblish.ValidateRenderSettings : AOV file format is not the same as the one set globally 1 != 6 // 

This is a known bug for Redshift Cryptomattes: #3410

---

// Error: pyblish.ValidateRenderSettings : AOV (Z) image prefix is not set correctly <BeautyPath>/<BeautyFile>.<RenderPass> != <BeautyPath>/<BeautyFile>_<RenderPass> // 

The others show that it's not being validated against the correct AOV separator.

That bug comes from the fact that it validates against instance.data["aovSeparator"] which is the aov separator actually being collected by the layer/AOV as collected in CollectRender and not the aov separator that the settings define.

As such, instead the validator should just be doing instance.data["aovSeparator"] != settings_aov_separator however that would technically be incorrect since Redshift allows a separator per AOV to be set instead of per layer and thus we should be validating that PER aov and not per layer. Thus even worse it shows that the collected aovSeparator in instance.data is wrong to begin with too. The separator isn't defined per layer/instance but per AOV and thus should be collected as such too.

As such this identifies an issue that might not be a simple fix - should we separate into an issue of its own? Or if not, if someone could take over this PR from here that'd be great :)

[antirotor]

should we separate into an issue of its own?

Yeah, it is not only redshift that allows separator to be defined per AOV (or rather the whole prefix, but that is question if we even want to support). So I vote for another issue as that one will be harder to solve. I thought we have one already for this but I cannot find it.

[m-u-r-p-h-y]

should this be merged than to get images file rule fixed?

[BigRoy]

I'd say so. This is a decent hotfix now for the issue at hand except for maybe this missing commit:

This commit would fix it.

• source comment

[m-u-r-p-h-y]

approved for images file rule