Bug: Houdini USD render publish instance errors on create

[BigRoy]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior:

It is currently impossible to create a USD Render (Experimental) instance as reported by @moonyuet here.

Traceback (most recent call last):
  File "S:\openpype\OpenPype\openpype\pipeline\create\context.py", line 1996, in _create_with_unified_error
    result = creator.create(*args, **kwargs)
  File "S:\openpype\OpenPype\openpype\hosts\houdini\plugins\create\create_usdrender.py", line 26, in create
    pre_create_data)  # type: CreatedInstance
  File "S:\openpype\OpenPype\openpype\hosts\houdini\api\plugin.py", line 191, in create
    self)
  File "S:\openpype\OpenPype\openpype\pipeline\create\context.py", line 914, in __init__
    data = copy.deepcopy(data or {})
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 281, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 241, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\PROGRA~1\SIDEEF~1\HOUDIN~1.435\python37\lib\copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle SwigPyObject objects

The issue is with this line:

        instance_data["parent"] = hou.node("/stage")

hou.Node instances can't be deep copied and thus can't be added to the instance data for the create method.
It'd need to be set as transient data afterwards if the reference would need to be directly added.

Expected Behavior:

Either:

1. Do not try to create the node in /stage and remove the logic.
2. Pass the parent where to create in in a different way to the parent class' create method.

Version

3.15.4-nightly.3

What platform you are running OpenPype on?

Windows

Steps To Reproduce:

1. Start Houdini
2. Create USD Render (experimental) in publisher.
3. Now you will have the error.

Are there any labels you wish to add?

• I have added the relevant labels to the bug report.

Relevant log output:

No response

Additional context:

No response

[cuID:OP-5765]

[BigRoy]

Expected Behavior:
Either:

1. Do not try to create the node in /stage and remove the logic.
2. Pass the parent where to create in in a different way to the parent class' create method.

@antirotor @moonyuet @fabiaserra

Which do we prefer? Keeping it in /out would be much more similar to how all other publishers work and maybe makes more sense?

[moonyuet]

Expected Behavior:
Either:

1. Do not try to create the node in /stage and remove the logic.
2. Pass the parent where to create in in a different way to the parent class' create method.

@antirotor @moonyuet @fabiaserra

Which do we prefer? Keeping it in /out would be much more similar to how all other publishers work and maybe makes more sense?

It makes sense for me as most of the instances(either render, review or vdb) set up their output nodes in '/out'.

[fabiaserra]

Hm, I could see that if we moved to a fully USD pipeline, we would probably rather stay within the native USD context of /stage instead but since we are still far from supporting that and it's just for outputting renders, it makes sense to stick to ROPs for it, but I see we might eventually want to evolve to using stage and publishing other families through it.

However, I'm confused with:

hou.Node instances can't be deep copied and thus can't be added to the instance data for the create method.

Why are we deep copying them?

[BigRoy]

Why are we deep copying them?

Any data in instance_data that the Creator receives on create goes through a deepcopy. In the end transientData ended up being used for it - I believe from this PR: #3897 (comment). In practice I think instance_data is supposed to be data that's basically JSON supported. @iLLiCiTiT can probably best explain it.

Couldn't we store the /stage string on "parent" instead, why do we need the hou.Node object?

In this case we definitely can just use the string representation since it's a static path anyway.

[iLLiCiTiT]

Any data in instance_data that the Creator receives on create goes through a deepcopy. In the end transientData ended up being used for it - I believe from this PR: #3897 (comment). In practice I think instance_data is supposed to be data that's basically JSON supported. @iLLiCiTiT can probably best explain it.

It is as Roy said. Instance data are data that should be stored to scene and should be possible to parse them using json. Any other information related to the instance should be stored to transient data.

[BigRoy]

This is fixed in Houdini - and USD publishing has had some very nice recent features added into it in AYON. I'll close this here.