Skip to content
This repository has been archived by the owner on Sep 20, 2024. It is now read-only.

Maya Deadline: Fix Tile Rendering by forcing integer pixel values #3758

Merged
merged 9 commits into from
Sep 14, 2022

Conversation

BigRoy
Copy link
Collaborator

@BigRoy BigRoy commented Aug 30, 2022

Brief description

This should resolve the issue faced in #3451

Description

Forces integer pixel values into the Tile Assembly config files.

Testing notes:

  1. Submit Tile Render
  2. Ensure tile render assembles correctly

@BigRoy BigRoy changed the title Force integer pixel values Maya Deadline: Fix Tile Rendering by forcing integer pixel values Aug 30, 2022
@m-u-r-p-h-y m-u-r-p-h-y added host: Maya module: Deadline AWS Deadline related features type: bug Something isn't working labels Aug 30, 2022
@m-u-r-p-h-y
Copy link
Member

There is an issue unrelated to this PR, but could be fixed here or should I create a new issue?

Error: RenderPluginException : GetBooleanPluginInfoEntryWithDefault: Could not parse value for key: RenderSetupIncludeLights into a boolean because: can not parse given string into a boolean value:  (Parameter 'str')
   at Deadline.Plugins.DeadlinePlugin.GetBooleanPluginInfoEntryWithDefault(String key, Boolean defaultValue) (Python.Runtime.PythonException)
  File "C:\ProgramData\Thinkbox\Deadline10\workers\WS01\plugins\630e05a37c5c5d5974843ee3\MayaBatch.py", line 293, in StartJob
    self.SetEnvironmentAndLogInfo( "MAYA_RENDER_SETUP_INCLUDE_ALL_LIGHTS", str( int( self.GetBooleanPluginInfoEntryWithDefault( "RenderSetupIncludeLights", True ) ) ) )
   at Python.Runtime.Dispatcher.Dispatch(ArrayList args)

@BigRoy
Copy link
Collaborator Author

BigRoy commented Aug 30, 2022

There is an issue unrelated to this PR, but could be fixed here or should I create a new issue?

@m-u-r-p-h-y could you show what value got submitted along with your job? Was the value the string None maybe?

It should be visible on the Job > Modify Job Properties > Submission Params > Plugin Info > RenderSetupIncludeLights=xxx?

@m-u-r-p-h-y
Copy link
Member

m-u-r-p-h-y commented Aug 30, 2022

@m-u-r-p-h-y could you show what value got submitted along with your job? Was the value the string None maybe?

It should be visible on the Job > Modify Job Properties > Submission Params > Plugin Info > RenderSetupIncludeLights=xxx?

yes, it is empty

image

and the project settings:

image

@BigRoy
Copy link
Collaborator Author

BigRoy commented Aug 30, 2022

@m-u-r-p-h-y could you test with this new commit?

By the way, that error should've only occurred with older Render instances in your scene that did not have the renderSetupIncludeLights attribute. So make sure to test with the same scene.

Copy link
Member

@m-u-r-p-h-y m-u-r-p-h-y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rendering is working fine in deadline

there is an error in OpenPypeTileAssembler though

Error: ValueError : invalid literal for int() with base 10: '540.0' (Python.Runtime.PythonException)
  File "C:\ProgramData\Thinkbox\Deadline10\workers\WS01\plugins\630e2a587c5c5d5974843eea\OpenPypeTileAssembler.py", line 335, in render_argument
    "height": int(data["Tile{}Height".format(tile)]),
   at Python.Runtime.Dispatcher.Dispatch(ArrayList args)
   at __FranticX_GenericDelegate0`1\[\[System_String\, System_Private_CoreLib\, Version=4_0_0_0\, Culture=neutral\, PublicKeyToken=7cec85d7bea7798e\]\]Dispatcher.Invoke()
   at FranticX.Processes.ManagedProcess.RenderArgument()
   at Deadline.Plugins.DeadlinePlugin.RenderArgument()
   at FranticX.Processes.ManagedProcess.Execute(Boolean waitForExit)
   at Deadline.Plugins.PluginWrapper.RenderTasks(Task task, String& outMessage, AbortLevel& abortLevel)

@m-u-r-p-h-y
Copy link
Member

m-u-r-p-h-y commented Aug 30, 2022

@m-u-r-p-h-y could you test with this new commit?

By the way, that error should've only occurred with older Render instances in your scene that did not have the renderSetupIncludeLights attribute. So make sure to test with the same scene.

I used the same scene for sure, rendering of tiles was successful, but tile assembly is not working now . .

@BigRoy
Copy link
Collaborator Author

BigRoy commented Aug 30, 2022

Ah sorry for wasting your time - I had missed some variables. However looking at this again I think this might have been a Py3 issue due to some divisions now returning float values - which also explains why this had worked in the past (py2-Maya-era). So I've forced an integer division now and reverted some of the changes.

Sorry to say it again, but if you have some time to try again - then thank you very much! 🙈

@m-u-r-p-h-y
Copy link
Member

m-u-r-p-h-y commented Aug 31, 2022

Submitting from Maya works, but there is still a couple of issues.

passing of offset values to oiio is not working, but I'm not sure if it is more suitable for standalone issue/PR or we can continue here?

2022-08-31 10:18:09:  0: INFO: Executable: "D:\REPO\OpenPype\vendor\bin\oiio\windows\oiiotool.exe"
2022-08-31 10:18:09:  0: PYTHON: MISSING IMPLEMENTATION: Unknown attrib type "float2". Value: 0, 0
2022-08-31 10:18:09:  0: PYTHON: MISSING IMPLEMENTATION: Unknown attrib type "float2". Value: 0, 0
2022-08-31 10:18:09:  0: PYTHON: MISSING IMPLEMENTATION: Unknown attrib type "float2". Value: 0, 0
2022-08-31 10:18:09:  0: PYTHON: MISSING IMPLEMENTATION: Unknown attrib type "float2". Value: 0, 0
2022-08-31 10:18:09:  0: PYTHON: MISSING IMPLEMENTATION: Unknown attrib type "float2". Value: 0, 0
2022-08-31 10:18:09:  0: INFO: Using arguments: --create:type=float 1920x1080 4 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+-540 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+-540 -o D:\PROJECTS\murphy_22101\shots\sh030\work\lighting\renders\maya\22101_sh030_workfileLighting_v010\Main\Main_beauty.1001.exr
2022-08-31 10:18:09:  0: INFO: Argument: --create:type=float 1920x1080 4 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+-540 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+-540 -o D:\PROJECTS\murphy_22101\shots\sh030\work\lighting\renders\maya\22101_sh030_workfileLighting_v010\Main\Main_beauty.1001.exr
2022-08-31 10:18:09:  0: INFO: Full Command: "D:\REPO\OpenPype\vendor\bin\oiio\windows\oiiotool.exe" --create:type=float 1920x1080 4 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+-540 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+-540 -o D:\PROJECTS\murphy_22101\shots\sh030\work\lighting\renders\maya\22101_sh030_workfileLighting_v010\Main\Main_beauty.1001.exr
2022-08-31 10:18:09:  0: INFO: Startup Directory: "D:\REPO\OpenPype\vendor\bin\oiio\windows"
2022-08-31 10:18:09:  0: INFO: Process Priority: BelowNormal
2022-08-31 10:18:09:  0: INFO: Process Affinity: default
2022-08-31 10:18:09:  0: INFO: Process is now running
2022-08-31 10:18:09:  0: STDOUT: oiiotool ERROR: --paste : Invalid offset '+0+-540'
2022-08-31 10:18:09:  0: STDOUT: Full command line was:
2022-08-31 10:18:09:  0: STDOUT: > oiiotool.exe --create:type=float 1920x1080 4 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_1x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +0+-540 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x2_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+0 D:/PROJECTS/murphy_22101/shots/sh030/work/lighting/renders/maya/22101_sh030_workfileLighting_v010/Main/_tile_2x1_2x2_Main_beauty.1001.exr --origin +0+0 --swap --paste +960+-540 -o D:\\PROJECTS\\murphy_22101\\shots\\sh030\\work\\lighting\\renders\\maya\\22101_sh030_workfileLighting_v010\\Main\\Main_beauty.1001.exr
2022-08-31 10:18:10:  0: INFO: Process exit code: 1

@BigRoy
Copy link
Collaborator Author

BigRoy commented Aug 31, 2022

Thanks - we could move it to another issue/PR but I think it's somewhat related to some changes I've made here.

2022-08-31 10:18:09:  0: STDOUT: oiiotool ERROR: --paste : Invalid offset '+0+-540'

That gets generated here and thus it looks like it finds a pos_y value that is negative value.

IF a negative value is required it should actually be +0-540 that could be done by reformatting that line of code to this:

            args.append("--paste {x:+d}{y:+d}".format(x=pos_x, y=pos_y))

I'll add a commit to do so.
However, I'm not sure negative offsets is what is to be expected for the Tile Assembler - especially since it doesn't look like this error was hit before.

This allows negative offsets to be pasted
@BigRoy
Copy link
Collaborator Author

BigRoy commented Aug 31, 2022

So I'm not sure about the tile formatting logic. Currently it does this:

width = 1920
height = 1080
tiles_x = 2
tiles_y = 2

w_space = width // tiles_x
h_space = height // tiles_y

for tile_x in range(1, tiles_x + 1):
    for tile_y in range(1, tiles_y + 1):
        
        print("--------")
        print("Tile(x={}, y={})".format(tile_x, tile_y))
        
        top = height - (tile_y * h_space)
        bottom = height - ((tile_y - 1) * h_space) - 1
        left = (tile_x - 1) * w_space
        right = (tile_x * w_space) - 1
        print(
f"""
    top
   {top:^5}
left     right
{left:^5}     {right:^5}
   bottom
   {bottom:^5}
""")

Resulting in e.g. this:

--------
Tile(x=1, y=1)

    top
    540 
left     right
  0        959 
   bottom
   1079 

--------
Tile(x=1, y=2)

    top
     0  
left     right
  0        959 
   bottom
    539 

--------
Tile(x=2, y=1)

    top
    540 
left     right
 960      1919 
   bottom
   1079 

--------
Tile(x=2, y=2)

    top
     0  
left     right
 960      1919 
   bottom
    539 

Which seem like odd numbers - no?

I'd have expected right to end on 960 and 1920?
Same for the bottom, that I had expected on 540 and 1080?

Like this:

width = 1920
height = 1080
tiles_x = 2
tiles_y = 2

w_space = width // tiles_x
h_space = height // tiles_y

for tile_x in range(1, tiles_x + 1):
    for tile_y in range(1, tiles_y + 1):
        
        print("--------")
        print("Tile(x={}, y={})".format(tile_x, tile_y))
        
        top = height - (tile_y * h_space)
        bottom = height - ((tile_y - 1) * h_space)
        left = (tile_x - 1) * w_space
        right = (tile_x * w_space)
        print(
f"""
    top
   {top:^5}
left     right
{left:^5}     {right:^5}
   bottom
   {bottom:^5}
""")

Either way, that doesn't explain how you got to a negative number like -540 in the first place. @m-u-r-p-h-y what was you resolution and amount of tiles?

@BigRoy
Copy link
Collaborator Author

BigRoy commented Aug 31, 2022

As mentioned up on OpenPype Discord the OpenPypeTileAssembler isn't working currently as it should. I'd say it might be best to leave that out of this PR - or keep this PR open until a decision is made on how to fix the full implementation of using Draft or the OpenPypeTileAssembler?

Let me know how you'd like me to proceed.

@iLLiCiTiT
Copy link
Member

Regarding deadline submit, the tile calculations are for Deadline Draft, which is now free (or should be?), so we should maybe change default tile renderer to Draft in settings.

To fix OpenPypeTileAssembler we would need more information. And it should be used not just in deadline, it should not use "Draft-like" data IMHO but more structured data model. To fully understand the requirements for OpenPypeTileAssembler we should have available rendered tiles from all possible renderers to compare their content. Known issues are different order or tiles (horizontal/vertical) or different x, y coordinate system. We should compare the rendered tiles and find out if we're able to assembly them in "unified way" or we need information about renderer or tiles.

@m-u-r-p-h-y
Copy link
Member

m-u-r-p-h-y commented Aug 31, 2022

I would suggest changing the default settings to Draft plugin in Deadline settings to make it usable.
OpenPype\openpype\settings\defaults\project_settings\deadline.json

Until we sort out the OpenPypeTileAssembler

@BigRoy can you add this change to this PR so we can merge it?

BigRoy added a commit to BigRoy/OpenPype that referenced this pull request Sep 8, 2022
Copy link
Member

@m-u-r-p-h-y m-u-r-p-h-y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tile rendering and Draft assembly work as expected

image

I would still recommend changing the defaults for deadline submit to Dratf (currently we default to OIIO) in openpype\settings\defaults\project_settings\deadline.json

publishing issue is tracked in standalone thread #3764

@BigRoy
Copy link
Collaborator Author

BigRoy commented Sep 12, 2022

Apparently I had secretly sneaked some of the fixes into the other PR as well - which now makes this PR mostly cosmetic tweaks. @antirotor Could you check whether I've done some weird things or not? :)

Copy link
Member

@antirotor antirotor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks fine, thanks!

@BigRoy
Copy link
Collaborator Author

BigRoy commented Sep 12, 2022

Be aware that this comment is still an open question. :)

@iLLiCiTiT
Copy link
Member

iLLiCiTiT commented Sep 14, 2022

Be aware that #3758 (comment) is still an open question. :)

I would say that question is related to OpenPypeTileAssembler logic. Not important in this PR at all.

EDITED: I just looked at changes here and those changes are related to OpenPypeTileAssembler 😄 . Is this PR relevant in that case?

@mkolar
Copy link
Member

mkolar commented Sep 14, 2022

I agree that the mentioned comment doesn't have relevance here and we can merge this.

@mkolar mkolar merged commit 045cbda into ynput:develop Sep 14, 2022
@BigRoy BigRoy deleted the fix_3451 branch March 20, 2024 15:21
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
host: Maya module: Deadline AWS Deadline related features type: bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants