Houdini local render support

[MustafaJafar]

Changelog Description

This PR adds support for local rendering in Houdini and marking render as reviewable.

Additional Info: How it works

The logic of this PR is based on re-using the collected data for each render instance to perform local rendering.
this happens by remapping the collected data expectedFiles where I use them to create a dedicated instance for each AOV in the expectedFiles while keeping the original instance empty.
I couldn't remove the original instance while this process because it holds the debug data.

• CollectFarmInstances
• Collect{..}RenderProducts (mantra, karma, redshift, arnold, vray)
  Compute expectedFiles which is a dictionary of AOVs and their files.
• CollectLocalRenderInstances
  if farm disabled append render.local.hou to families
  create instance for each AOV in expected files and add essential data and its representation data
• ExtractLocalRender
  if farm disabled, if skip_render disabled, render the node
• Deadline stuff should work only if farm is enabled.

Testing notes:

1. Open Houdini
2. Create Render instance while disabling Submitting to Farm
3. Publish (it should work)
4. Create Render instance while enabling Submitting to Farm
5. Publish (it should work as usual)

[MustafaJafar]

Can we make this an EnumDef with options that say:

Local Render
Farm Render

That way we can expand it later to also include:

Use Existing Frames (local)
Use Existing Frames (on farm)

Which is something I blelieve Nuke and Fusion also do. Would love to hear other options though.

That way the skip_render toggle is basically part of this combox

For reference here's how it looks like

[MustafaJafar]

So far this PR works but I have some considerations from my point of view:

1. Using the instance node as the source of truth: which can no longer happen because the exposed creator_attributes can change without updating the instance node itself.
   So, collectors won't the collect the correct data.
   Currently, I use creator_attributes as the source of truth in collect render product plugins and use an extractor plugin to modify the Houdini node parameters accordingly.
   This also limits what creator_attributes that can be exposed in publish tab.
   Therefore, I created Houdini: Sync publisher attributes with node parameters #418 as an attempt to use instance node as the source of truth by syncing instance nodes and publisher.

2. Marking instance as reviewable, It works on farm and local but it also runs review plugins dedicated to opengl node. This problem is shown here https://github.com/ynput/ayon-core/pull/328/files#r1558145584
   Currently, I solved it temporarily by adding if rop_node.type().name() != "opengl" in the unrelated plugins.

3. Creating new instances: tbh, I'm not sure if I should use a dedicated function to create instances.
   Currently, I create them and fill the necessary data keys manually. It works so far.

[moonyuet]

I tested with local machine rendering, use existing frames and farm rendering withs single AOVrender.
So far so good

[moonyuet]

I tested with AOV renders in Mantra. Looks good. Maybe there should be some reminders that each renderer has different AOV suffix in terms of filenames, and we won't misuse the $AOV into the renderers other than into Redshift.

[antirotor]

Tested both local and farm rendering with Redshift and it works. I am still not convinced with farm_split name as it's not very descriptive and then there is splitRender used. But I have no idea how to change it :)

[MustafaJafar]

I am still not convinced with farm_split name as it's not very descriptive and then there is splitRender used.

I couldn't pick better names.
I'd appreciate any suggestions.

[moonyuet]

Should we merge this ticket or we still need to work on something else? @MustafaJafar @antirotor

[krishnaavril]

Can we also have a feature to publish through the Solaris tree renders? Currently, the karma rop is working well. but, all the people use only the solaris tree as we have more control there on stage. Please let me know your thoughts on this.

[BigRoy]

Can we also have a feature to publish through the Solaris tree renders? Currently, the karma rop is working well. but, all the people use only the solaris tree as we have more control there on stage. Please let me know your thoughts on this.

Are you saying you would like to publish or submit using the USD Render ROP inside /stage/.. and not a USD Render ROP in /out?

For what it's worth, I believe #295 implements supporting USD Render ROP for Deadline submissions using Husk standalone for rendering - however, I think it's still aimed at working in /out.

I'm working on some more houdini-wide efforts to support more generalized publishing and submissions that is a more global 'rethink' of how things work in Houdini. This may affect also your question here over time.

Could you elaborate on your exact use case?