fix for string attributes default values

[benblo]

• see attributes' default value not saved in scene #34
• handles Compounds recursively

[mottosso]

Let's add a docstring test to this, something that..

1. Creates this attribute
2. Saves the scene
3. Opens it to confirm the value remains

[benblo]

I'm not aware of docstring tests, is that a type of unit-test? is there a specific syntax?
Would that test do?

# new scene with a sphere with "my_string" attribute
cmds.file(new=True, force=True)
sphere_name = cmds.sphere()[0]
sphere = cmdx.encode(sphere_name)
sphere.add_attr(cmdx.String("my_string", default="foo"))

# save scene
scene_path = "test.ma"
cmds.file(rename=scene_path)
cmds.file(save=True, type="mayaAscii")

# reload scene & check "my_string"
cmds.file(scene_path, open=True, force=True)
sphere = cmdx.encode(sphere_name)
print(sphere["my_string"])
assert sphere["my_string"] == "foo"

[mottosso]

Any function/method with a >>> in it will be run as part of the test suite. That example looks fine, put that as an example in the docstring to a relevant function and it'll get run for every commit/PR/push.

Anything following a >>> is what is compared against to determine whether the test succeeds.

>>> 5 == 5
False

This test translates into assert((5 == 5) == False) and would fail.

>>> var = 5
>>> var
5

This test succeeds. Etc.

[benblo]

Got it, I added the test.

[mottosso]

Great, the tests have replied. :)

https://mottosso.visualstudio.com/cmdx/_build/results?buildId=889&view=logs&j=5b9df9af-0fac-585a-7539-c045100ff967&t=dbc97956-5956-52a2-6392-afdf28434b48&l=205

[benblo]

Great, the tests have replied. :)

Ahahah awesome :), this is a great CI pipeline you got there!
Looks like every test needs to import cmdx, didn't know.

Still failing though. I understand Maya2015's failure (seems to not find the plug, don't know why), but I don't understand it for Maya2020: it's re-opening the scene and then... nothing? https://mottosso.visualstudio.com/cmdx/_build/results?buildId=890&view=logs&j=fba502cc-81c4-5a7b-0b6c-e040a5b5b330&t=10abd179-32b1-54ec-572c-7ab220df8c4d&l=169

[benblo]

Oooh I see, it's failing right at the start!

Expected nothing, got "untitled"

I'm supposed to tell it the expected result of everything single step? I must be missing something...
Figured it out, functions that return a value must be either validated or turned into dummy assignments.

[benblo]

Well, I'm at a loss... the test runs fine on my Maya, but it looks like it's not finding the plug: plug.name() returns nurbsSphere1., I would expect nurbsSphere1.my_string here (?)... so "my_string" is missing, could this be related to plug reuse somehow? maybe the flag isn't in the same state during tests vs my Maya?

[mottosso]

Failed example:
    sphere["my_string"]
Expected:
    "foo"
Got:
    foo

Some Maya's are less forgiving and more secretive in their error messages.

• https://mottosso.visualstudio.com/cmdx/_build/results?buildId=892&view=logs&j=2b237643-a38b-526e-4678-34891bde6218&t=85c198ea-7c27-5294-1374-9805cc42010e&l=144

Keep in mind that what's being printed there is a stringification of sphere["my_string"] which will be an attribute, not a string. You might want to either str(sphere["my_string"]) or sphere["my_string"].read().

[benblo]

Well, there you have it:

Failed example:
    sphere["my_string"].name()
Expected:
    'my_string'
Got:
    ''

Passes the first time, fails after scene reload (line 5156).
Is there any way to get at the test artifacts? ie the test.ma file that is generated, so I can look at what Maya saves.

AFAICT this is the first test in the codebase that tries to save & reload a file so I must be hitting some shenanigan here...

[mottosso]

Passes the first time, fails after scene reload (line 5156).

Haha, oh wow. Wouldn't like to be you right now. xDD

Is there any way to get at the test artifacts? ie the test.ma file that is generated, so I can look at what Maya saves.

Print it. :)

if fail:
  with open("somescene.ma") as f:
    print(f.read())

That'll give you the contents in the test page.

There are probably other ways in which to upload artifacts in the CI system for download later, but I bet that's a rabbit hole.

[mottosso]

I hope this is not a more serious issue of node-reuse thinking that nodes from a past scene are accessible from the newly opened scene. I would expect memory violations from that. It really shouldn't, as the uniqueness comes from their hash which should be as unique as their memory address.

[benblo]

Well, the "my_string" attribute is indeed there, so shenanigan it is...
Is there a way to kill the plug cache or something?

(I actually have had some crashes in the context of destroying/recreating a bunch of nodes with the same name, which I felt could be due to node/plug reuse... I was hoping to get to those subjects once the pleasantries were done...)

[benblo]

Hmm... so some tests pass, some don't, seems very random.
I wonder if it could it be due to the fact that the tests run in parallel, so eg Maya 2015 and 2018 are writing to the same file concurrently? do you have one agent per Maya version, or is it all on the same server? (if "server" is even still a thing with Azure...)
Now trying with a tempfile...

[benblo]

Yay, one success... well, I think this makes it pretty clear that the attribute is indeed saved (just grep "my_string" in the failure cases, it's in the .ma for sure), but the node/plug is invalid. I'm gonna try to disable caching and see what happens.

[benblo]

Well, that didn't help, now 2020 is the only one passing (was 2016 last time)... this feels very random.
I'm officially out of ideas, gonna move on for now.

[benblo]

So I just figured out that I didn't set the CMDX_SAFE_MODE env var correctly (needs to be "True/False", "1/0" doesn't work...)
... suddenly Maya 2018 and up pass! lower versions still all fail :(

[mottosso]

Sorry you have to go through this, debugging CI is a pain.

You can run these locally though, so don't forget that. Either if you've got Docker installed locally, or by heading to https://labs.play-with-docker.com/

You'll find commands you can use from the CI config here which you can likely put into a Bash script or the like to speed things up.

[benblo]

Aaah, Docker... I'm on Windows here man ;) !
I actually have only the vaguest of idea of what Docker is, but I remember it from a brief stint at a CG studio where the name kept popping up; it was a proper all-Linux environment, as I then learned is the standard in the CG world... but I'm a gamedev myself.
Anyway, I don't have a clue how to host any of this, building VMs etc, so replicating your setup is probably not a trivial task for me. I wish though, looks neat!

If I push commits to my own fork (without a PR), does that trigger CI as well?

[mottosso]

I'm on Windows as well. :) Docker runs on Windows, and also on WSL2. But, you don't need any of that, the play-with-docker website let's you launch the Docker container - which is like launching a terminal into another machine - used by the CI from your browser, where you can call commands as though it was your local machine. And you won't need to replicate any setup beyond having access to the docker command, that's what Docker is for; encapsulating an environment - OS and all - in a big binary file.

Once you're done pulling your hair out fighting CI, take 10 mins to run through a tutorial. You won't regret it.

If I push commits to my own fork (without a PR), does that trigger CI as well?

No, but you're welcome to make PRs to trigger builds, even if you don't intend to merge them. Just put something in the description to avoid mistakenly merging it.

[benblo]

So... I can just tell some docker somewhere in the cloud to pull a fully-featured VM image, with Maya installed etc? What about licenses?

[mottosso]

Can't merge this one until tests are happy.

[mottosso]

@benblo I'm also having a few issues with crashes where Maya isn't giving you much intel on why. I'm curious about your experience in the past few months, whether you've got any new insights or debugging tricks? This crashing issue still eludes me personally, and I have a few that I think may be related but are having trouble narrowing down.

[benblo]

Crashes have become much, much less frequent for me (my deleteAttr fix has definitely paid off), now I'm only having the odd crash when switching scenes, which is still annoying but not workflow-disrupting so I haven't even tried investigating yet.
I'm afraid I don't have much insight, I'm a bit baffled by Maya's crashdumps, sometimes it generates a Python callstack (which is, for my use, much more informative than the C++ dumps), but sometimes it doesn't and I don't understand why. The fact that they don't release pdbs is really lame.
I haven't tried piping the console output to a file like you described, it's ridiculous as well that it isn't built-in (Unity does this for example, always has). It should be always on so you have info after the fact, not "hey I'm supposed to know I'm likely to crash so I need to setup a pipe beforehand", ugh.

I've been working on UI a lot lately, and Qt just loooves to crash when you give it badly typed data (eg pass a float to a setText(), or even None, and bam!), it's pretty lame that shiboken doesn't type-check. We use pybind on our own lib at work, and I don't think I've ever had a crash, pybind just asserts all the inputs.
So after untold crashes like this, I just made a subset lib of Qt where I just cast all the inputs to the proper type, it's pretty handy as well, you can pass anything to a label and it just str()s it as you expect in Python, as print() would.

I definitely need to PR some more fixes your way, but I've been buried for over a month figuring out some insane transform issues in some of our rigs, which I finally nailed yesterday so yay! I can finally move on to other stuff.
BTW, I noticed something super weird where getting node.transform(sWorld, time=0) gives incorrect result compared with node.transform(sWorld, time=None) when the node has a parent constraint (so position is a driven key), even though the position isn't animated/keyed... I really should open an issue for this, though I'm not sure it has anything to do with cmdx, but maybe you know if it's expected or not.

[mottosso]

So after untold crashes like this, I just made a subset lib of Qt where I just cast all the inputs to the proper type

Yes, I'm having similar thoughts about cmdx, of making a compiled version where we expose critical parts of the API myself rather than relying on maya.api. At least that way we could (1) assert inputs and (2) get a proper callstack when things do crash. I suspect the C++ API is more battle hardened than the Python bindings anyway. We could even use pybind and maybe get a few sanity checks for free.

I definitely need to PR some more fixes your way

Awesome, that would be great. I'm working on a too-large PR at the moment with everything and the kitchen sink in it - #52. I'm just struggling with the Azure pipeline no longer working with Docker (apparently) so am moving to GitHub Actions. Another CI platform, another file syntax, and still no means of testing the script before pushing. It's a lot of fun (not). xD

BTW, I noticed something super weird where getting node.transform(sWorld, time=0) gives incorrect result compared with node.transform(sWorld, time=None)

Incorrect how? Wouldn't 0 give you the value at time 0, and None give you the value at the current time?

[mottosso]

In fact I suspect the primary reason for crashes on my end are strings. Creating them, editing them, deleting them. How is your current experience with strings? Are you using them a lot?

[mottosso]

@benblo I just discovered that there are debug symbols for Maya! :OOOO

They are not public, they are hosted under the Autodesk Developer Network, which is free. So yeah, sign up for that. It's a solid 5GB of debug symbols. Just giving these a go now.

[benblo]

Regarding validating marshalling (Qt, OpenMaya, etc): overall, typing is the #1 thing that I hate about Python. I'm constantly guessing what functions want, casting stuff, printing stuff just to see what their type is; so infuriating when statically-typed languages solved this issue 50 years ago.
I think discovering Python with Maya probably makes this 10 times worse, I keep reading articles about "good" Python software using proper type hints (Py 3), and static analyzers like MyPy that make it much closer to working with a proper compiler; being stuck on a 10 years old Python release is laughable/cryable to begin with.

But on top of that, Autodesk did a very shoddy job with the Python bindings, so much of the API is exposed as bla(*args, *kwargs). I don't know how much of it is due to my IDE (VsCode), I've tried many things, using the completion info from the devkit, or using a VsCode extension (that apparently is itself generating typing info from the docs). So I constantly have to go to the online docs just to understand the method argument type & order. Don't even get me started on cmds, with its numbnuts idea of "one function that does 50 loosely related but very different things depending on the arguments".
Same issue with Qt, I can either use the PySide2 typeshed, which has good info but is missing eg the signals, or the devkit completion package, which is complete but doesn't have method arguments... pick your half-assed poison.
Again, super frustrating since both the OpenMaya and Qt docs are very extensive and unambiguous. I keep telling myself I need to set aside a few days to just grep it all (maybe they're even available somewhere in .rst format, or something more parsable than html) and spit out a correct (and complete!) .pyi package. But why oh why isn't this provided out of the box...

BTW, this is also why I've added # type: hints to cmdx (I have more of those locally that I need to PR).
The math wrappers are a little half-baked too though, as for example cdmx.Plug.asTransformationMatrix() returns a cmdx.TransformationMatrix, but asEulerRotation() returns an om.MEulerRotation. Or, cmdx.Vector + cmdx.Vector returns an om.MVector: that's just weird.
We're constantly oscillating between cmdx and om; I kinda think if we're gonna wrap the math types, then we should wrap all the way... I'm just not sure how bad the overhead would be.
It would also (for me) solve autocompletion, so I could for example chain calls, like xfo.rotation().inverse(): at the moment cmdx.TransformationMatrix.rotation() isn't wrapped so it uses the om.MTransformationMatrix.rotation(), which has no type info, so VsCode thinks it returns None and doesn't offer the .inverse() completion.

---

Re string crashes: yes I'm using them. On every scene I have a dozen nodes with a couple strings attributes I'm frequently reading from/writing to, I haven't seen issues with it. I have that fix locally though, so maybe that helps.

---

Re symbols: hmm... pretty sure ADN is where I downloaded the devkit to begin with, how did I miss the symbols?!? You mean there right: https://adn.autodesk.io/ ? Now I can't seem to log on, that's weird...

---

BTW, I noticed something super weird where getting node.transform(sWorld, time=0) gives incorrect result compared with node.transform(sWorld, time=None)

Incorrect how? Wouldn't 0 give you the value at time 0, and None give you the value at the current time?

That's the thing, that node isn't keyed so its value is the same at t0 vs tCurrent.

[mottosso]

Don't even get me started on cmds, with its numbnuts idea of "one function that does 50 loosely related but very different things depending on the arguments".

Haha, yes. Having implemented a few of those via their provided C++ API, I can see why they do that. Each command has a ton of boilerplate associated with it, and you basically want to squeeze as much into each command as you can. Using arguments as sub-commands. It's very aweful. But it goes back to the MEL days, which is still how scenes are serialised to disk. Python was added later and this is one area which doesn't lend itself well to it.

The math wrappers are a little half-baked too though, as for example cdmx.Plug.asTransformationMatrix() returns a cmdx.TransformationMatrix, but asEulerRotation() returns an om.MEulerRotation. Or, cmdx.Vector + cmdx.Vector returns an om.MVector: that's just weird.

That is true, they should return "native" cmdx types. PR is welcome.

Re symbols: hmm... pretty sure ADN is where I downloaded the devkit to begin with, how did I miss the symbols?!? You mean there right: https://adn.autodesk.io/ ? Now I can't seem to log on, that's weird...

Yes, hah. It's very well hidden. Known as a "Special Download".

• https://adn.autodesk.io/index.php?r=custom_pages%2Fview&id=25

[benblo]

They are not public, they are hosted under the Autodesk Developer Network, which is free. So yeah, sign up for that. It's a solid 5GB of debug symbols. Just giving these a go now.

Hi! I'm reviving this topic :), trying to solidify our Maya pipeline here... I still can't figure out how to get an ADN account!
You say it's free, but all I can find is either "ADN Open", that doesn't actually seem to require any registration, but also does not provide any special access, and "ADN Standard", which is definitely not free (or "ADN Professional", even more expensive).
Do you remember how you got your account?

[mottosso]

Haha, a little out of scope for a GitHub PR, but it's the ADN Standard here.

• https://www.autodesk.com/developer-network/membership-options

The secret is in the finetext below.

Startup developers qualify to receive ADN Standard membership at no charge to support their startup projects and software development efforts.

If you've got a company/studio, and you develop for Maya, this is you. :)

[mottosso]

Closing this for now as the main branch has deviated too far from this PR. Feel free to reopen this if a solution is found, it's still an active problem.